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PREFACE 


This textbook is an expanded version of Elementary Linear Algebra, Ninth Edition, by Howard Anton. The first ten chapters of 
this book are identical to the first ten chapters of that text; the eleventh chapter consists of 21 applications of linear algebra 
drawn from business, economics, engineering, physics, computer science, approximation theory, ecology, sociology, 
demography, and genetics. The applications are, with one exception, independent of one another and each comes with a list of 
mathematical prerequisites. Thus, each instructor has the flexibility to choose those applications that are suitable for his or her 
students and to incorporate each application anywhere in the course after the mathematical prerequisites have been satisfied. 

This edition of Elementary Linear Algebra, like those that have preceded it, gives an elementary treatment of linear algebra that 
is suitable for students in their freshman or sophomore year. The aim is to present the fundamentals of linear algebra in the 
clearest possibleway; pedagogy is the main consideration. Calculus is not a prerequisite, but there are clearly labeled exercises 
and examples for students who have studied calculus. Those exercises can be omitted without loss of continuity. Technology is 
also not required, but for those who would like to use matlab, Maple, Mathematica, or calculators with linear algebra 
capabilities, exercises have been included at the ends of the chapters that allow for further exploration of that chapter's 
contents. 


SUMMARY OF CHANGES 
IN THIS EDITION 


This edition contains organizational changes and additional material suggested by users of the text. Most of the text is 
unchanged. The entire text has been reviewed for accuracy, typographical errors, and areas where the exposition could be 
improved or additional examples are needed. The following changes have been made: 


Section 6.5 has been split into two sections: Section 6.5 Change of Basis and Section 6.6 Orthogonal Matrices. This 
allows for sharper focus on each topic. 


A new Section 4.4 Spaces of Polynomials has been added to further smooth the transition to general linear 
transformations, and a new Section 8.6 Isomorphisms has been added to provide explicit coverage of this topic. 


Chapter 2 has been reorganized by switching Section 2.1 with Section 2.4. The cofactor expansion approach to 
determinants is now covered first and the combinatorial approach is now at the end of the chapter. 


Additional exercises, including Discussion and Discovery, Supplementary, and Technology exercises, have been added 
throughout the text. 


In response to instructors' requests, the number of exercises that have answers in the back of the book has been reduced 
considerably. 


The page design has been modified to enhance the readability of the text. 


A new section on the earliest applications of linear algebra has been added to Chapter 11. This section shows how linear 
equations were used to solve practical problems in ancient Egypt, Babylonia, Greece, China, and India. 



Hallmark Features 


Relationships Between Concepts One of the important goals of a course in linear algebra is to establish the intricate 
thread of relationships between systems of linear equations, matrices, determinants, vectors, linear transformations, and 
eigenvalues. That thread of relationships is developed through the following crescendo of theorems that link each new 
idea with ideas that preceded it: 1.5.3, 1.6.4, 2.3.6, 4.3.4, 5.6.9, 6.2.7, 6.4.5, 7.1.5. These theorems bring a coherence to 
the linear algebra landscape and also serve as a constant source of review. 


Smooth Transition to Abstraction The transition from R n to general vector spaces is often difficult for students. To 
smooth out that transition, the underlying geometry of R n is emphasized and key ideas are developed in R n before 
proceeding to general vector spaces. 


Early Exposure to Linear Transformations and Eigenvalues To ensure that the material on linear transformations 
and eigenvalues does not get lost at the end of the course, some of the basic concepts relating to those topics are 
developed early in the text and then reviewed and expanded on when the topic is treated in more depth later in the text. 

For example, characteristic equations are discussed briefly in the chapter on determinants, and linear transformations from 
R H to R m are discussed immediately after R n is introduced, then reviewed later in the context of general linear 
transformations. 


About the Exercises 

Each section exercise set begins with routine drill problems, progresses to problems with more substance, and concludes with 
theoretical problems. In most sections, the main part of the exercise set is followed by the Discussion and Discovery problems 
described above. Most chapters end with a set of supplementary exercises that tend to be more challenging and force the 
student to draw on ideas from the entire chapter rather than a specific section. The technology exercises follow the 
supplementary exercises and are classified according to the section in which we suggest that they be assigned. Data for these 
exercises in matlab, Maple, and Mathematica formats can be downloaded from www.wiley.com/college/anton . 

About Chapter 11 

This chapter consists of 21 applications of linear algebra. With one clearly marked exception, each application is in its own 
independent section, so that sections can be deleted or permuted freely to fit individual needs and interests. Each topic begins 
with a list of linear algebra prerequisites so that a reader can tell in advance if he or she has sufficient background to read the 
section. 

Because the topics vary considerably in difficulty, we have included a subjective rating of each topic—easy, moderate, more 
difficult. (See “A Guide for the Instructor” following this preface.) Our evaluation is based more on the intrinsic difficulty of 
the material rather than the number of prerequisites; thus, a topic requiring fewer mathematical prerequisites may be rated 
harder than one requiring more prerequisites. 

Because our primary objective is to present applications of linear algebra, proofs are often omitted. We assume that the reader 
has met the linear algebra prerequisites and whenever results from other fields are needed, they are stated precisely (with 
motivation where possible), but usually without proof. 

Since there is more material in this book than can be covered in a one-semester or one-quarter course, the instructor will have 
to make a selection of topics. Help in making this selection is provided in the Guide for the Instructor below. 

Supplementary Materials for Students 

Student Solutions Manual, Ninth Edition —This supplement provides detailed solutions to most theoretical exercises and to at 
least one nonroutine exercise of every type. (ISBN 0-471-43329-2) 



Data for Technology Exercises is provided in matlab, Maple, and Mathematica formats. This data can be downloaded from 
w w w. wiley. com/college/anton . 


Linear Algebra Solutions —Powered by JustAsk! invites you to be a part of the solution as it walks you step-by-step through a 
total of over 150 problems that correlate to chapter materials to help you master key ideas. The powerful online 
problem-solving tool provides you with more than just the answers. 

Supplementary Materials for Instructors 

Instructor's Solutions Manual —This new supplement provides solutions to all exercises in the text. (ISBN 0-471-44798-6) 

Test Bank —This includes approximately 50 free-form questions, five essay questions for each chapter, and a sample 
cumulative final examination. (ISBN 0-471-44797-8) 

eGrade —eGrade is an online assessment system that contains a large bank of skill-building problems, homework problems, 
and solutions. Instructors can automate the process of assigning, delivering, grading, and routing all kinds of homework, 
quizzes, and tests while providing students with immediate scoring and feedback on their work. Wiley eGrade “does the 
math”... and much more. For more information, visit http://www.wiley.com/college/egrade or contact your Wiley 
representative. 

Web Resources —More information about this text and its resources can be obtained from your Wiley representative or from 
www.wiley.com/college/anton . 


A GUIDE FOR THE 
INSTRUCTOR 


Linear algebra courses vary widely between institutions in content and philosophy, but most courses fall into two categories: 
those with about 35^-0 lectures (excluding tests and reviews) and those with about 25-30 lectures (excluding tests and 
reviews). Accordingly, I have created long and short templates as possible starting points for constructing a course outline. In 
the long template I have assumed that all sections in the indicated chapters are covered, and in the short template I have 
assumed that instructors will make selections from the chapters to fit the available time. Of course, these are just guides and 
you may want to customize them to fit your local interests and requirements. 

The organization of the text has been carefully designed to make life easier for instructors working under time constraints: A 
brief introduction to eigenvalues and eigenvectors occurs in Sections 2.3 and 4.3, and linear transformations from R n to R m are 
discussed in Chapter 4. This makes it possible for all instructors to cover these topics at a basic level when the time available 
for their more extensive coverage in Chapters 7 and 8 is limited. Also, note that Chapter 3 can be omitted without loss of 
continuity for students who are already familiar with the material. 



Long Template 

Short Template 

Chapter 1 

7 lectures 

6 lectures 

Chapter 2 

4 lectures 

3 lectures 

Chapter 4 

4 lectures 

4 lectures 

Chapter 5 

7 lectures 

6 lectures 

Chapter 6 

6 lectures 

3 lectures 











Long Template 

Short Template 

Chapter 7 

4 lectures 

3 lectures 

Chapter 8 

6 lectures 

2 lectures 

Total 

38 lectures 

27 lectures 


Variations in the Standard Course 

Many variations in the long template are possible. For example, one might create an alternative long template by following the 
time allocations in the short template and devoting the remaining 11 lectures to some of the topics in Chapters 9, 10 and 11. 

An Applications-Oriented Course 

Once the necessary core material is covered, the instructor can choose applications from Chapter 9 or Chapter 11. The 
following table classifies each of the 21 sections in Chapter 11 according to difficulty: 

Easy. The average student who has met the stated prerequisites should be able to read the material with no help from the 
instructor. 


Moderate. The average student who has met the stated prerequisites may require a little help from the instructor. 


More Difficult. The average student who has met the stated prerequisites will probably need help from the instructor. 


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 


EASY • • 


MODERATE 


MORE 

DIFFICULT 
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CHAPTER 


Systems of Linear Equations and Matrices 


NTRODUCTlOh Information in science and mathematics is often organized into rows and columns to form rectangular arrays, 
called “matrices” (plural of “matrix”). Matrices are often tables of numerical data that arise from physical observations, but they also 
occur in various mathematical contexts. For example, we shall see in this chapter that to solve a system of equations such as 

5x+j = 3 
2*-j=4 

all of the information required for the solution is embodied in the matrix 

5 1 3' 

_ 2 -1 4_ 

and that the solution can be obtained by performing appropriate operations on this matrix. This is particularly important in 
developing computer programs to solve systems of linear equations because computers are well suited for manipulating arrays of 
numerical information. However, matrices are not simply a notational tool for solving systems of equations; they can be viewed as 
mathematical objects in their own right, and there is a rich and important theory associated with them that has a wide variety of 
applications. In this chapter we will begin the study of matrices. 
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1.1 

INTRODUCTION TO 
SYSTEMS OF LINEAR 
EQUATIONS 


Systems of linear algebraic equations and their solutions constitute one of the 
major topics studied in the course known as “linear algebra. ” In this first section 
we shall introduce some basic terminology and discuss a method for solving such 
systems. 


Linear Equations 

Any straight line in the xy-plane can be represented algebraically by an equation of the form 

a\x H ct^y = b 

where a\, a 2 - and b are real constants and a\ and a 2 are not both zero. An equation of this form is called a linear equation in the 
variables x and y. More generally, we define a linear equation in the n variables rj, ij, .x„ to be one that can be expressed in the 
form 

a\x\ 4- ayx 2 H-1- a H x M = b 

where a\, a 2 - ■■■, and b are real constants. The variables in a linear equation are sometimes called unknowns. 


EXAMPLE 1 Linear Equations 


The equations 


x-f3y = 7, y = ^x + 3z±l, and xj — 2^2 — 3x3 + x 4 = 7 

are linear. Observe that a linear equation does not involve any products or roots of variables. All variables occur only to the first 
power and do not appear as arguments for trigonometric, logarithmic, or exponential functions. The equations 

x 4- 3\[y = 5, 3x -I- 2y — z 4 xz = 4, and i y = sinx 


are not linear. 


A solution of a linear equation a\x\ 4 - ayxj H- V a„x H = b is a sequence of n numbers sp S 2 , such that the equation is 

satisfied when we substitute x 1 = si, X 2 = S2, x n =s n . The set of all solutions of the equation is called its solution set or 

sometimes the general solution of the equation. 


Finding a Solution Set 


Find the solution set of (a) 4x — 2y — and (b) x j _ 4^2 4 7x3 = 5- 


Solution (a) 

To find solutions of (a), we can assign an arbitrary value to x and solve for y, or choose an arbitrary value for y and solve for x. If 
we follow the first approach and assign x an arbitrary value t, we obtain 

x = £, y = 2t -i 

These formulas describe the solution set in terms of an arbitrary number t, called a parameter. Particular numerical solutions can be 







obtained by substituting specific values for t. For example, t = 3 yields the solution x = 3, y = 


11 


and t = — yields the solution 



If we follow the second approach and assign y the arbitrary value t, we obtain 

’'=!*+!• y= l 

Although these formulas are different from those obtained above, they yield the same solution set as t varies over all possible real 
numbers. For example, the previous formulas gave the solution x = 3, y = when t = 3, whereas the formulas immediately above 

yield that solution when t = 


Solution (b) 


To find the solution set of (b), we can assign arbitrary values to any two variables and solve for the third variable. In particular, if 
we assign arbitrary values 5 and t to xj and X3, respectively, and solve for xj, we obtain 

x 1 = 5 4 4 s — It, X 2 — s, X3 —t 


♦ 


Linear Systems 


A finite set of linear equations in the variables ..., x H is called a system of linear equations or a linear system. A sequence 

of numbers sj, 53, ..., s n is called a solution of the system if xj = sj, 13 = si, ..., x„ = s n is a solution of every equation in the 
system. For example, the system 

4xi — X 2 4 3x3 = — 1 
3xi 4 X 2 4 9x3 = — 4 

has the solution jq = 1, X2 = 2 > X3 = — 1 since these values satisfy both equations. However, x j = ], *3 = 8> X3 = 1 is not a 
solution since these values satisfy only the first equation in the system. 


Not all systems of linear equations have solutions. For example, if we multiply the second equation of the system 

*4 y = 4 
2 x 2 y = 6 


by 7 ^, it becomes evident that there are no solutions since the resulting equivalent system 


r4j = 4 
x 4 y = 3 


has contradictory equations. 


A system of equations that has no solutions is said to be inconsistent ; if there is at least one solution of the system, it is called 
consistent. To illustrate the possibilities that can occur in solving systems of linear equations, consider a general system of two 
linear equations in the unknowns x and y: 

a ix 4 b\y = c\ (a 1, A1 not both zero) 

«2X 4 A2V = C2 («2> A2 not both zero) 


The graphs of these equations are lines; call them f and / 2 - Since a point (x, v) lies on a line if and only if the numbers x and v 
satisfy the equation of the line, the solutions of the system of equations correspond to points of intersection of ^ and l 2 - There are 
three possibilities, illustrated in Figure 1.1.1: 


Eii) No solution 




(O In finiLely m tiny so In \ ions 

Figure 1.1.1 


The lines f and / 2 may be parallel, in which case there is no intersection and consequently no solution to the system. 

The lines f and / 2 may intersect at only one point, in which case the system has exactly one solution. 

The lines f and / 2 may coincide, in which case there are infinitely many points of intersection and consequently infinitely 
many solutions to the system. 


Although we have considered only two equations with two unknowns here, we will show later that the same three possibilities hold 
for arbitrary linear systems: 


Every system of linear equations has no solutions, or has exactly one solution, or has infinitely many solutions. 


An arbitrary system of m linear equations in n unknowns can be written as 


an*! + ^12*2 H-1- = b i 

<321*1 4 13 22*2 4 - + ^2h*h = ^2 


fl ml* 1 4 <3 m 2*2 b — 4 


fl ™*H — & 










where jq, ..., * H are the unknowns and the subscripted a' s and b's denote constants. For example, a general system of three 
linear equations in four unknowns can be written as 

isiiari 4 <212*2 4 <*13*3 4 <*14*4 = &1 
<* 21*1 4 <* 22*2 4 < 323*3 4 < 324*4 = *2 
<331*1 + <*32*2 4 <333*3 + <*34*4 = 63 


The double subscripting on the coefficients of the unknowns is a useful device that is used to specify the location of the coefficient 
in the system. The first subscript on the coefficient fly indicates the equation in which the coefficient occurs, and the second 
subscript indicates which unknown it multiplies. Thus, a 12 is in the first equation and multiplies unknown 

Augmented Matrices 

If we mentally keep track of the location of the +'s, the x's, and the ='s, a system of m linear equations in n unknowns can be 
abbreviated by writing only the rectangular array of numbers: 


AH 

<*12 ■ 


*1 

<*21 

<*22 ■ 

" a 2 n 

i >2 

<*ml 

<*m2 ' 




This is called the augmented matrix for the system. (The term matrix is used in mathematics to denote a rectangular array of 
numbers. Matrices arise in many contexts, which we will consider in more detail in later sections.) For example, the augmented 
matrix for the system of equations 

*1 V *2 4 2*3 = 9 
2*i I 4*2 — 3*3 — 1 
3 *i f 6*2 — 5*3 = 0 
is 

'1 1 29 ' 

2 4 — 3 1 

3 6-50 


Remark When constructing an augmented matrix, we must write the unknowns in the same order in each equation, and the 
constants must be on the right. 


The basic method for solving a system of linear equations is to replace the given system by a new system that has the same solution 
set but is easier to solve. This new system is generally obtained in a series of steps by applying the following three types of 
operations to eliminate unknowns systematically: 


Multiply an equation through by a nonzero constant. 


2. Interchange two equations. 


3, Add a multiple of one equation to another. 


Since the rows (horizontal lines) of an augmented matrix correspond to the equations in the associated system, these three 
operations correspond to the following operations on the rows of the augmented matrix: 


Multiply a row through by a nonzero constant. 






2, Interchange two rows. 


3. Add a multiple of one row to another row. 

Elementary Row Operations 

These are called elementary row operations. The following example illustrates how these operations can be used to solve systems 
of linear equations. Since a systematic procedure for finding solutions will be derived in the next section, it is not necessary to 
worry about how the steps in this example were selected. The main effort at this time should be devoted to understanding the 
computations and the discussion. 


Using Elementary Row Operations 


In the left column below we solve a system of linear equations by operating on the equations in the system, and in the right column 
we solve the same system by operating on the rows of the augmented matrix. 


x + y + 2z = 9 
2x + Ay-3z=\ 

3x + 6y — 5z = 0 

Add -2 times the first equation to the second to obtain 


11 2 9 

2 4-31 

3 6-50 


Add -2 times the first row to the second to obtain 


x + y + 2z = 

9 


'1 

1 2 

9' 


II 

1 

& 

- 17 


0 

2 -7 

- 17 


3x + 6 y —5z — 

0 


3 

6 -5 

0 


Add -3 times the first equation to the third to obtain 

Add -3 times the first row to the 

third to obtain 

x + y 4 - 2 z = 

9 


'1 

1 2 

9' 


II 

£ 

1 

& 

- 17 


0 

2 -7 

- 17 


1 

ET 

II 

-27 


0 

3-11 

-27 


Multiply the second equation by to obtain 

Multiply the second row by -T to 

obtain 

x - 1 - y 4 - 2 z = 

9 


'l 

1 2 

9 


II 

Mcm 

1 

17 

2 


0 

’ -I 

17 

2 


II 

23 

1 

& 

-27 


0 

3-11 

-27 


Add -3 times the second equation to the third to obtain 

Add -3 times the second row to the third to obtain 

x -\-y + 2z = 

9 


"l 

1 2 

9" 


y- 2 z=- 

2 

17 

2 


0 


17 

2 


-!z = 

3 


0 

o -I 

3 


2 

2 



2 

2 


Multiply the third equation by - 2 to obtain 

Multiply the third row by -2 to obtain 














x + y + 2 z = 

9 

'l 

1 

2 

9' 


17 

2 

0 

1 

7 

2 

17 

2 

z = 

3 

0 

0 

1 

3 


Add -1 times the second equation to the first to obtain 


Add -1 times the second row to the first to obtain 


x 



1 0 

0 1 
0 0 


li 

2 

_ 7 

2 

1 


35 

2 

11 

' 2 

3 


11 7 

Add — -y- times the third equation to the first and times the 

third equation to the second to obtain 

x = 1 

7 =2 
7 = 3 

The solution x = \, y = 2, z = 3 is now evident. 


11 7 

Add — —— times the third row to the first and A times the 
2 2 

third row to the second to obtain 

"10 0 1 “ 

0 10 2 
0 0 13 


Exercise Set 1.1 




Click here for Just Ask! 


1 . 


Which of the following are linear equations in jq, xj, and 13 ? 


4 5x2 ~ /2ij = 1 

(b) x\ 4- 3i2 +1113 = 2 

(c) x\ = — 1 x 2 + 3*3 

^ i , -2 + *2 + 8*3 = 5 
ij ^ — 2i2 4 *3=4 
© **i-/2*2 + }*3=7 1/3 


2. 


Given that k is a constant, which of the following are linear equations? 










(a) * 1—*2 I 7l "3 = sink 


(b) * 11 - 1*2 = 9 


(c) 2^*i 4 7*2 — *3 = 0 


Find the solution set of each of the following linear equations. 

3. 


(a) 7* — 5.y = 3 


(b) 3*i — 5*2 4 4*3 = 7 


(c) — 8 *i-| 2*2 — 5*3 I 6 * 4=1 


(d) 3v - Sw 4 2 *-y I Az = 0 


Find the augmented matrix for each of the following systems of linear equations. 

4. 


(a) 3*i — 2*2 = — 1 
4*i 4 5*2 = 3 

7*i 4 3*2 = 2 


(b) 2 *i A 2 * 3=1 
3*i — *2 4 4*3 = 7 
6 *i 4 *2 — *3 = 0 


(c) *i H 2*2 — * 4 d *5 = 1 

3*2 I *3 —*5 = 2 

*3 b 7*4 =1 


(d) *i = 1 
*2 =2 
*3 = 3 


5. 


Find a system of linear equations corresponding to the augmented matrix. 





(a) 


2 

3 

0 


0 0 
-4 0 
1 1 


(b) 


3 

7 

0 


0 

1 

-2 



5 

-3 

7 


(c) 


7 2 1 
1 2 4 


-3 5 

0 1 


(d) 


10 0 0 
0 10 0 
0 0 10 
0 0 0 1 


7 

-2 

3 

4 


6 . 

(a) Find a linear equation in the variables x and y that has the general solution * = 5 + 2 t,y = t- 


(b) Show that x = U y = 2^~2 ^ a ' so t * ie 8 enera ' solution of the equation in part (a). 


The curve y = ax 2 f bx + c shown in the accompanying figure passes through the points (;q, 71 ), (x 2 , 72 )’ ar *d (* 3 , 73 )- Show 
that the coefficients a, b, and c are a solution of the system of linear equations whose augmented matrix is 

x\ xi 1 71 

x 2 *2 l yi 

X 2 x 3 1 73 



Consider the system of equations 

x + y 4 - 2 z = a 
x + z = b 
2x 4 - y H 3 z = c 

Show that for this system to be consistent, the constants a , b , and c must satisfy c = a + b- 














Show that if the linear equations x\+kx 2 = c and x\ + 1x2 —d ^ ave ^ ie same solution set, then the equations are identical. 


9. 

Show that the elementary row operations do not affect the solution set of a linear system. 

10 . 


Discussion 

Discovery 


n. 


For which value(s) of the constant k does the system 

x - y = 3 
2x — 2y=k 

have no solutions? Exactly one solution? Infinitely many solutions? Explain your reasoning. 


Consider the system of equations 

12 . 


ax 4- by — k 
cx + dy = 1 
ex + fy = m 


Indicate what we can say about the relative positions of the lines ax \ by = k- cx + dy = i, and 
ex + f y = m when 


(a) the system has no solutions. 


(b) the system has exactly one solution. 


(c) the system has infinitely many solutions. 


If the system of equations in Exercise 12 is consistent, explain why at least one equation can be 

13. discarded from the system without altering the solution set. 

If k = 1 = m = ij in Exercise 12, explain why the system must be consistent. What can be said about 

14. the point of intersection of the three lines if the system has exactly one solution? 

We could also define elementary column operations in analogy with the elementary row operations. 

15. What can you say about the effect of elementary column operations on the solution set of a linear 
system? How would you interpret the effects of elementary column operations? 
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I n In this section we shall develop a systematic procedure for solving systems of 

linear equations. The procedure is based on the idea of reducing the augmented 
GAUSSIAN ELIMINATION matrix of a system to another augmented matrix that is simple enough that the 

solution of the system can be found by inspection. 


Echelon Forms 

In Example 3 of the last section, we solved a linear system in the unknowns x, y, and z by reducing the augmented matrix to the 
form 

'10 0 1 ' 

0 10 2 
0 0 13 

from which the solution % = 1, y = 2,z = 3 became evident. This is an example of a matrix that is in reduced row-echelon form. To 
be of this form, a matrix must have the following properties: 


If a row does not consist entirely of zeros, then the first nonzero number in the row is a 1. We call this a leading 1. 

2, If there are any rows that consist entirely of zeros, then they are grouped together at the bottom of the matrix. 

In any two successive rows that do not consist entirely of zeros, the leading 1 in the lower row occurs farther to the right than 
the leading 1 in the higher row. 

Each column that contains a leading 1 has zeros everywhere else in that column. 


A matrix that has the first three properties is said to be in row-echelon form. (Thus, a matrix in reduced row-echelon form is of 
necessity in row-echelon form, but not conversely.) 


Row-Echelon and Reduced Row-Echelon Form 


The following matrices are in reduced row-echelon form. 


1 0 0 
0 1 0 
0 0 1 


1 

'1 

0 

o' 



0 

1 

0 


- 

0 

0 

1 



0 

0 

0 

0 


1 

0 

0 

0 


The following matrices are in row-echelon form. 


-2 0 1 
0 1 3 
0 0 0 
0 0 0 


0 0 
0 0 


'1 

4 

-3 

7' 


'1 

1 

o' 


'0 

1 

2 

6 

O' 

0 

1 

6 

2 


0 

1 

0 


0 

0 

1 

- 1 

0 

0 

0 

1 

5 


0 

0 

0 


0 

0 

0 

0 

1 


We leave it for you to confirm that each of the matrices in this example satisfies all of the requirements for its stated form. 





















EXAMPLE 2 More on Row-Echelon and Reduced Row-Echelon Form 


As the last example illustrates, a matrix in row-echelon form has zeros below each leading 1, whereas a matrix in reduced 
row-echelon form has zeros below and above each leading 1. Thus, with any real numbers substituted for the *'s, all matrices of the 
following types are in row-echelon form: 


} * * * 

0 1** 
0 0 1 * 

0 0 0 1 

] * * * 

0 1 ** 

0 0 0 0 

0 0 0 0 


1 * 
0 1 
0 0 
0 0 

0 1 
0 0 
0 0 
0 0 
0 0 


* 

* 

1 

0 


0 

0 

0 

0 


* 

* 

* 

0 


1 * 
0 1 
0 0 
0 0 


* * 

* * 

* * 

1 * 

0 0 


* * 

* * 

* * 

* * 

0 1 


* 

* 

* 

* 

* 


Moreover, all matrices of the following types are in reduced row-echelon form: 


'l 

0 

0 

o" 


'l 

0 

0 

" 

0 

1 

0 

0 


0 

1 

0 

* 

0 

0 

1 

0 


0 

0 

1 

* 

0 

0 

0 

1 


0 

0 

0 

0 


1 0 
0 1 
0 0 
0 0 


* 

* 

0 

0 


* 

* 

0 

0 


01 * 000 * * 0 * 
000100 * * 0 * 
000010 * * 0 * 
000001 * * 0 * 
0000000 01 * 


♦ 


If, by a sequence of elementary row operations, the augmented matrix for a system of linear equations is put in reduced row-echelon 
form, then the solution set of the system will be evident by inspection or after a few simple steps. The next example illustrates this 
situation. 


Solutions of Four Linear Systems 


Suppose that the augmented matrix for a system of linear equations has been reduced by row operations to the given reduced 
row-echelon form. Solve the system. 


(a) 


10 0 5 
0 10-2 
0 0 1 4 


1004-1 
0 10 2 6 

0 0 13 2 


(b) 


























(c) 


1 6 0 0 4 -2 
0 0 1 0 3 1 
0 0 0 1 5 2 
0 0 0 0 0 0 


(d) 


1 

0 

0 


0 0 0 
1 2 0 
0 0 1 


Solution (a) 

The corresponding system of equations is 

*1 =5 

xj = - 2 

X2 = 4 

By inspection, x t = 5 , X2 = _ 2, x 3 = 4- 

Solution (b) 

The corresponding system of equations is 

*1 +4*4= —1 

*2 1 2*4 = 6 

*3 4 3*4 = 2 

Since * 2 , and *3 correspond to leading l's in the augmented matrix, we call them leading variables or pivots. The nonleading 
variables (in this case * 4 ) are called free variables. Solving for the leading variables in terms of the free variable gives 

*1 = — 1 — 4*4 
*2 = 6 — 2*4 
*3 = 2 — 3*4 

From this form of the equations we see that the free variable *4 can be assigned an arbitrary value, say t , which then determines the 
values of the leading variables *p *2, and * 3 . Thus there are infinitely many solutions, and the general solution is given by the 
formulas 

*l = — 1 — 4;, *2 = 6 — 2 1 , *3 = 2 — 3 1 , *4 = t 


Solution (c) 


The row of zeros leads to the equation o*! + 0*2 I 0*3 1 0*4 1 0*5 = 0> which places no restrictions on the solutions (why?). 

Thus, we can omit this equation and write the corresponding system as 

*1 + 6*2 b4*5 = — 2 

*3 + 3 * 5 = 1 

*4 + 5*5 = 2 

Here the leading variables are * 1 , * 3 , and * 4 , and the free variables are *2 and * 5 . Solving for the leading variables in terms of the 
free variables gives 

*1 = — 2 — 6*2 — 4*5 
*3=1— 3*5 
*4=2 — 5*5 


Since *5 can be assigned an arbitrary value, t , and *2 can be assigned an arbitrary value, 5 , there are infinitely many solutions. The 
general solution is given by the formulas 


*1 = — 2 — 8 s — 4t r 


*2 = s , 


*3 =1—3 1, 


*4 = 2 — 5f, 


* \ = l 






Solution (d) 

The last equation in the corresponding system of equations is 

0*1 I- 0x2 + 0*3 = 1 

Since this equation cannot be satisfied, there is no solution to the system. 


Elimination Methods 

We have just seen how easy it is to solve a system of linear equations once its augmented matrix is in reduced row-echelon form. 
Now we shall give a step-by-step elimination procedure that can be used to reduce any matrix to reduced row-echelon form. As \ 
state each step in the procedure, we shall illustrate the idea by reducing the following matrix to reduced row-echelon form. 

'0 0 -2 0 7 12' 

2 4 - 10 6 12 28 

2 4 -5 6 -5 -1 


Step Locate the leftmost column that does not consist entirely of zeros. 


0 

0 

-2 

0 

7 

12 

2 

4 

- 10 

6 

12 

28 

2 

4 

-5 

6 

-5 

- 1 


Leftmost nonzero rohunn 


Step 2 Interchange the top row with another row, if necessary, to bring a nonzero entry to the top of the column found in Step 1. 


2 

4 

- 10 

6 

12 

28' 


0 

0 

-2 

0 

7 

12 

<— The first and second rows in the 

2 

4 

-5 

6 

-5 

- 1 

preceding matrix were interchanged. 


Step . If the entry that is now at the top of the column found in Step 1 is a, multiply the first row by 1 la in order to introduce a 
leading 1 . 


1 2 - 5 3 6 14 

0 0 -2 0 7 12 

2 4 -5 6 -5 -1 


The first row of the preceding 
matrix was multiplied by 


Step 4 Add suitable multiples of the top row to the rows below so that all entries below the leading 1 become zeros. 


1 2 -5 3 6 14 

0 0 -2 0 7 12 

0 0 5 0 - 17 -29 


<-2 times the first row of the preceding 

matrix was added to the third row. 


Step L Now cover the top row in the matrix and begin again with Step 1 applied to the submatrix that remains. Continue in this 
way until the entire matrix is in row-echelon form. 












1 

2 

— 5 

3 

6 

14 

0 

0 

-2 

0 

7 

12 

0 

0 

5 

0 

- 17 

-29 


Leftmost nonzero column in the snhmotiix 


1 

2 

-5 

3 

6 

14 


0 

0 

1 

0 

_ 7 

9 

— 

6 

+— The first row in the submatrix was 

_0 

0 

5 

0 

jLi 

- 17 

— 29_ 

multiplied by — ^ to introduce a leading 1 

'l 

2 

-5 

3 

6 

14 



0 

0 

1 

0 

_ 7 

-6 


^-5 times the first row of the submatrix 

0 

0 

0 

0 

1 

1 


was added to the second row of the 

2 


submatrix to introduce a zero 


below the leading 1. 


'i 

2 

-5 

3 

6 

14" 

0 

0 

1 

0 

7 

2 

-6 

0 

0 

0 

0 

1 

2 

1 


The top row in the submatrix was 
covered, and we returned again to Step 1. 


L 


Leftmost nonzero tolmtm in the new submatiix 


1 2 - 5 3 6 14 

00 1 0 -1 -6 

2 

0 0 0 0 1 2 


The first (and only) row in the new 
submatrix was multiplied by 2 
to introduce a leading 1. 


The entire matrix is now in row-echelon form. To find the reduced row-echelon form we need the following additional step. 


Step 6 Beginning with the last nonzero row and working upward, add suitable multiples of each row to the rows above to introduce 
zeros above the leading l's. 


1 2 - 5 3 6 14 

0 0 1 0 0 1 

0 0 0 0 1 2 

"1 2 - 5 3 0 2" 

0 0 1 0 0 1 

0 0 0 0 1 2 

" 120307 " 

0 0 1 0 0 1 

0 0 0 0 1 2 

The last matrix is in reduced row-echelon form. 

If we use only the first five steps, the above procedure produces 
the procedure through to the sixth step and producing a matrix ii 


7 

*— — times the third row of the preceding 
matrix was added to the second row. 

*— — 6 times the third row was 
added to the first row. 

5 times the second row was 
added to the first row. 


a row-echelon form and is called Gaussian elimination. Carrying 
reduced row-echelon form is called Gauss-Jordan elimination. 


It can be shown that every matrix has a unique reduced row-echelon form; that is, one will arrive at the same reduced 
row-echelon form for a given matrix no matter how the row operations are varied. (A proof of this result can be found in the article 
“The Reduced Row Echelon Form of a Matrix Is Unique: A Simple Proof,” by ThomasYuster, Mathematics Magazine , Vol. 57, No. 
2, 1984, pp. 93-94.) In contrast, a row-echelon form of a given matrix is not unique : different sequences of row operations can 
produce different row-echelon forms. 


















Karl Friedrich Gauss 


Karl Friedrich Gauss (1777-1855) was a German mathematician and scientist. Sometimes called the “prince of 
mathematicians,” Gauss ranks with Isaac Newton and Archimedes as one of the three greatest mathematicians who ever lived. In 
the entire history of mathematics there may never have been a child so precocious as Gauss—by his own account he worked out 
the rudiments of arithmetic before he could talk. One day, before he was even three years old, his genius became apparent to his 
parents in a very dramatic way. His father was preparing the weekly payroll for the laborers under his charge while the boy 
watched quietly from a corner. At the end of the long and tedious calculation, Gauss informed his father that there was an error 
in the result and stated the answer, which he had worked out in his head. To the astonishment of his parents, a check of the 
computations showed Gauss to be correct! 


In his doctoral dissertation Gauss gave the first complete proof of the fundamental theorem of algebra, which states that every 
polynomial equation has as many solutions as its degree. At age 19 he solved a problem that baffled Euclid, inscribing a regular 
polygon of seventeen sides in a circle using straightedge and compass; and in 1801, at age 24, he published his first masterpiece, 
Disquisitiones Arithmeticae, considered by many to be one of the most brilliant achievements in mathematics. In that paper 
Gauss systematized the study of number theory (properties of the integers) and formulated the basic concepts that form the 
foundation of the subject. Among his myriad achievements, Gauss discovered the Gaussian or “bell-shaped” curve that is 
fundamental in probability, gave the first geometric interpretation of complex numbers and established their fundamental role in 
mathematics, developed methods of characterizing surfaces intrinsically by means of the curves that they contain, developed the 
theory of conformal (angle-preserving) maps, and discovered non-Euclidean geometry 30 years before the ideas were published 
by others. In physics he made major contributions to the theory of lenses and capillary action, and with Wilhelm Weber he did 
fundamental work in electromagnetism. Gauss invented the heliotrope, bifilar magnetometer, and an electrotelegraph. 

Gauss, who was deeply religious and aristocratic in demeanor, mastered foreign languages with ease, read extensively, and 
enjoyed mineralogy and botany as hobbies. He disliked teaching and was usually cool and discouraging to other mathematicians, 
possibly because he had already anticipated their work. It has been said that if Gauss had published all of his discoveries, the 
current state of mathematics would be advanced by 50 years. He was without a doubt the greatest mathematician of the modern 


Wilhelm Jordan 


Wilhelm Jordan (1842-1899) was a German engineer who specialized in geodesy. His contribution to solving linear systems 
appeared in his popular book, Handbuch der Vermessungskunde (Handbook of Geodesy), in 1888. 


EXAMPLE 4 Gauss-Jordan Elimination 


Solve by Gauss-Jordan elimination. 


+ 3*2 — 2*3 4 - 2x$ 

2x\ + 6x2 ~ 5*3 — 2x4 4-4x5 — 3 x$ - 
5x2 + 10*4 + 15*0 = 

2xj+ 6x2 I 8x4 + 4 xj + 18 x 0 = 


0 

- 1 

5 

6 


Solution 


The augmented matrix for the system is 


1 3 

-2 

0 

2 

0 

0 

2 6 

-5 

-2 

4 

-3 

- 1 

0 0 

5 

10 

0 

15 

5 

2 6 

0 

8 

4 

18 

6 

urth rows gives 




1 3 

-2 

0 

2 

0 

0 

0 0 

- 1 

-2 

0 

-3 

- 1 

0 0 

5 

10 

0 

15 

5 

0 0 

4 

8 

0 

18 

6 


Multiplying the second row by -1 and then adding -5 times the new second row to the third row and -4 times the new second row 
to the fourth row gives 


1 

3 

-2 

0 

2 

0 

0 

0 

0 

1 

2 

0 

3 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

6 

2 










Interchanging the third and fourth rows and then multiplying the third row of the resulting matrix by -y gives the row-echelon form 

6 


1 3 -2 0 2 0 0 

0 0 1 2 0 3 1 

0 0 0 0 0 1 I 

3 

0 0 0 0 0 0 0 


Adding -3 times the third row to the second row and then adding 2 times the second row of the resulting matrix to the first row 
yields the reduced row-echelon form 


1 3 0 4 2 0 0 

0 0 1 2 0 0 0 

0 0 0 0 0 1 I 

3 

0 0 0 0 0 0 0 


The corresponding system of equations is 


x\ 4- 3x2 - 1 - 4 x 4 + 2 xj 

*3 + 2 x 4 


-0 
= 0 


*6 


1_ 

3 


(We have discarded the last equation, 0 xi 4 . 0 x 3 4- 0*3 + 0 x 4 + Ox 5 + 0x^ = 0, since it will be satisfied automatically by the 
solutions of the remaining equations.) Solving for the leading variables, we obtain 


xi = — 3x2 — 4x4 — 2x^ 


X 3 = — 2 x 4 



If we assign the free variables X 2 > X 4 , and x^ arbitrary values r, 5 , and f, respectively, the 

x\ = —3r— 4s— 2t, X 2 = r, X 3 = — 2 s, X 4 = s. 


general solution is given by the formulas 
x 5 = t, x 6 = i 


♦ 


Back-Substitution 

It is sometimes preferable to solve a system of linear equations by using Gaussian elimination to bring the augmented matrix into 
row-echelon form without continuing all the way to the reduced row-echelon form. When this is done, the corresponding system of 
equations can be solved by a technique called back-substitution. The next example illustrates the idea. 


Example 4 Solved by Back-Substitution 


From the computations in Example 4, a row-echelon form of the augmented matrix is 


13-20200 
0 0 1 2 0 3 1 

00 0 0 0 1 1 

3 

00 00000 


To solve the corresponding system of equations 

xi + 3x2 


— 2 x 3 

X 3 + 2 x 4 


I 2 x 5 


= 0 


I 3x6 = 1 



we proceed as follows: 

Step 1 Solve the equations for the leading variables. 










x\ = — 3^2 H 27:3 — 2x$ 
7:3 =1 — 27:4 — - ,x 6 



Step 2 Beginning with the bottom equation and working upward, successively substitute each equation into all the equations above 
it. 


— 3t:2 4 27:3 — 2ttj5 

— 27:4 

l 

3 

— 3x2 ~ 4*4 — 2x$ 

— 2 x 4 

l 

3 


Substituting — A. into the second equation yields 

*1 = 
7:3 = 

*6 = - 

Substituting * 3 = — 2*4 into the first equation yields 

*1 = 
7:3 = 

*6 — 


Step Assign arbitrary values to the free variables, if any. 

If we assign 7:3,7:4, and x$ the arbitrary values r, s , and t , respectively, the general solution is given by the formulas 
X\= -3r-4s-2t, X 2 = r, xj= -2 s, x 4 = s, x$ = t, x$ = ^ 

This agrees with the solution obtained in Example 4. 

♦ 


Remark The arbitrary values that are assigned to the free variables are often called parameters. Although we shall generally use 
the letters r, s, t, ... for the parameters, any letters that do not conflict with the variable names may be used. 


Gaussian Elimination 


Solve 


by Gaussian elimination and back-substitution. 


x+ y 4 2z = 9 
2x + 4y — 3z = 1 
3x + by — 5z = 0 


Solution 

This is the system in Example 3 of Section 1.1. In that example we converted the augmented matrix 

'1 1 29 ' 

2 4-31 

3 6-50 


1 1 2 


0 0 1 


9 

II 

2 

3 


to the row-echelon form 








The system corresponding to this matrix is 


Solving for the leading variables yields 


x + y + 2z = 



z = 


9 

17 

2 

3 


x = 9-y-2z 

y = — — + —z 
■* 2 2 

z=3 


Substituting the bottom equation into those above yields 

* = 3-7 
7 = 2 
z = 3 

and substituting the second equation into the top yields x = 1, y = 2, z = 3- This agrees with the result found by Gauss-Jordan 
elimination in Example 3 of Section 1.1. 


Homogeneous Linear Systems 

A system of linear equations is said to be homogeneous if the constant terms are all zero; that is, the system has the form 

<311*1 + <312*2 H-f <31 h* m = 0 

<321*1 + <322*2 H-1 a 2 m*m = 0 

tf™l*l f«m2*2 H-1 «™*h = 0 

Every homogeneous system of linear equations is consistent, since all such systems have x j = 0, X 2 = 0’ •••,?;„ = 0 as a solution. 

This solution is called the trivial solution ; if there are other solutions, they are called nontrivial solutions. 

Because a homogeneous linear system always has the trivial solution, there are only two possibilities for its solutions: 

The system has only the trivial solution. 


The system has infinitely many solutions in addition to the trivial solution. 


In the special case of a homogeneous linear system of two equations in two unknowns, say 

a\x + b\y = Q [a\, h\ not both zero) 

«2* + = 0 t«2' &2 both zero) 

the graphs of the equations are lines through the origin, and the trivial solution corresponds to the point of intersection at the origin 
(Figure 1.2.1). 


in) Only ihi: trivial solution 



{A) Infinitely many solutions 

Figure 1.2.1 

There is one case in which a homogeneous system is assured of having nontrivial solutions—namely, whenever the system involves 
more unknowns than equations. To see why, consider the following example of four equations in five unknowns. 


Gauss-Jordan Elimination 


Solve the following homogeneous system of linear equations by using Gauss-Jordan elimination. 


2xi H 2x2— x 3 + xj = 0 

— xi — X2 + 2x3 — 3 x 4 + Xj; = 0 

xi + X2 — 2x3 — x; = 0 

X 3 +X 4 + x^ = 0 

Solution 

The augmented matrix for the system is 

‘2 2-1 0 10 

-1-1 2-3 10 

1 1-2 0-10 

0 0 1 1 10 

Reducing this matrix to reduced row-echelon form, we obtain 


1 

1 

0 

0 

1 

0 

0 

0 

1 

0 

1 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 


The corresponding system of equations is 












*l+*2 +*;5 = 0 

7; 3 +7:5 — 0 

X 4 = 0 


Solving for the leading variables yields 

* 3 = ~X5 

X 4 — 0 

Thus, the general solution is 

X\=—S — t, X7=S, *3 = — t, *4=0, 

Note that the trivial solution is obtained when s = t = 0. 


x*, = t 


( 2 ) 


♦ 


Example 7 illustrates two important points about solving homogeneous systems of linear equations. First, none of the three 
elementary row operations alters the final column of zeros in the augmented matrix, so the system of equations corresponding to the 
reduced row-echelon form of the augmented matrix must also be a homogeneous system [see system 2], Second, depending on 
whether the reduced row-echelon form of the augmented matrix has any zero rows, the number of equations in the reduced system 
is the same as or less than the number of equations in the original system [compare systems 1 and 2], Thus, if the given 
homogeneous system has m equations in 11 unknowns with m < and if there are r nonzero rows in the reduced row-echelon form 
of the augmented matrix, we will have r < n . It follows that the system of equations corresponding to the reduced row-echelon form 
of the augmented matrix will have the form 

■■**! +H() = o 

^ ^0 = ° (3 ) 

’ x kr +H() = 0 

where x kv x k2 , x kr are the leading variables and H() denotes sums (possibly all different) that involve the n _ r free variables 

[compare system 3 with system 2 above]. Solving for the leading variables gives 

Xfc! = -E() 

*k 2 = -H() 

*k r = -H() 

As in Example 7, we can assign arbitrary values to the free variables on the right-hand side and thus obtain infinitely many solutions 
to the system. 

In summary, we have the following important theorem. 


THEOREM 1.2.1 


A homogeneous system of linear equations with more unknowns than equations has infinitely many solutions. 


Remark Note that Theorem 1.2.1 applies only to homogeneous systems. A nonhomogeneous system with more unknowns than 
equations need not be consistent (Exercise 28); however, if the system is consistent, it will have infinitely many solutions. This will 
be proved later. 

Computer Solution of Linear Systems 

In applications it is not uncommon to encounter large linear systems that must be solved by computer. Most computer algorithms 
for solving such systems are based on Gaussian elimination or Gauss-Jordan elimination, but the basic procedures are often 
modified to deal with such issues as 



Reducing roundoff errors 


Minimizing the use of computer memory space 


Solving the system with maximum speed 


Some of these matters will be considered in Chapter 9. For hand computations, fractions are an annoyance that often cannot be 
avoided. However, in some cases it is possible to avoid them by varying the elementary row operations in the right way. Thus, once 
the methods of Gaussian elimination and Gauss-Jordan elimination have been mastered, the reader may wish to vary the steps in 
specific problems to avoid fractions (see Exercise 18). 


Remark Since Gauss-Jordan elimination avoids the use of back-substitution, it would seem that this method would be the more 
efficient of the two methods we have considered. 

It can be argued that this statement is true for solving small systems by hand since Gauss-Jordan elimination actually involves less 
writing. However, for large systems of equations, it has been shown that the Gauss-Jordan elimination method requires about 50% 
more operations than Gaussian elimination. This is an important consideration when one is working on computers. 


Exercise Set 1.2 



Click here for Just Ask! 


Which of the following 3x3 matrices are in reduced row-echelon form? 

1 . 


(a) 


1 0 0 
0 1 0 
0 0 1 


(b) 


1 0 0 
0 1 0 
0 0 0 


(c) 


0 1 0 
0 0 1 
0 0 0 


(d) 


1 0 0 
0 0 1 
0 0 0 


(e) 


1 0 0 
0 0 0 
0 0 1 














(f) 


0 1 0 
1 0 0 
0 0 0 


(g) 


1 1 0 
0 1 0 
0 0 0 


(h) 


1 0 2 
0 1 3 
0 0 0 


(i) 


0 0 1 
0 0 0 
0 0 0 


(j) 


0 0 0 
0 0 0 
0 0 0 


Which of the following 3x3 matrices are in row-echelon form? 

2 . 


(a) 


1 0 0 
0 1 0 
0 0 1 


(b) 


1 2 0 
0 1 0 
0 0 0 


(c) 


1 0 0 
0 1 0 
0 2 0 


(d) 


1 3 4 
0 0 1 
0 0 0 


(e) 


15-3 
0 1 1 

0 0 0 




















(f) 


1 2 3 
0 0 0 
0 0 1 


In each part determine whether the matrix is in row-echelon form, reduced row-echelon form, both, or neither. 

3. 


(a) 


1 2 0 3 0 
0 0 110 
0 0 0 0 1 
0 0 0 0 0 


(b) 


10 0 5 
0 0 13 
0 10 4 


(c) 


10 3 1 
0 12 4 


(d) 


1 

0 


-7 5 5 
1 3 2 


(e) 


1 3 0 2 0 
1 0 2 2 0 
0 0 0 0 1 
0 0 0 0 0 


(f) 


0 0 
0 0 
0 0 


In each part suppose that the augmented matrix for a system of linear equations has been reduced by row operations to the givei 
reduced row-echelon form. Solve the system. 


(a) 


10 0-3 
0 10 0 

0 0 1 7 


100-7 8 

0 10 3 2 

001 1-5 


(b) 









(c) 


1 - 6003-2 
0 0 1 0 4 7 

0 0 0 1 5 8 

0 0 0 0 0 0 


(d) 


1-300 
0 0 10 

0 0 0 1 


In each part suppose that the augmented matrix for a system of linear equations has been reduced by row operations to the given 
5. row-echelon form. Solve the system. 


(a) 


1 

0 

0 


-3 4 7 
1 2 2 
0 1 5 


(b) 


108-56 
014-93 
0 0 1 12 


(c) 


1 7 
0 0 
0 0 
0 0 


-2 0 -8 -3 

11 6 5 

0 1 3 9 

0 0 0 0 


(d) 


1 

0 

0 


-3 7 1 
1 4 0 
0 0 1 


Solve each of the following systems by Gauss-Jordan elimination. 

6 . 


(a) xi 4- *2 -+ 2^3 = 8 

— xi — 2x2 + 3x3 = 1 
3xj — 7x2 -1-4x3 = 10 

(b) 2xi +- 2x2 + 2x3 = 0 

— 2xi +5x2 +2x3= 1 

8 xi + X2 4- 4x3 = — 1 

(c) x — y ±2z— w=—1 
2 x -¥ y — 2z — 2w — — 2 

-x-\-2y — 4z f w = 1 

3x - 3w = - 3 











(d) — 2A +- 3c — 1 

3 a \- 6 b — 3c = — 2 
6 a I 6 b -h 3c = 5 


Solve each of the systems in Exercise 6 by Gaussian elimination. 

7. 


Solve each of the following systems by Gauss-Jordan elimination. 

8 . 


(a) 2 xi — 3 x 2 = — 2 

2xi + -2 — 1 
3xi + 2x2 = 1 


(b) 3xi b 2x2 — *3 = — 15 

5xi b 3x2 b 2x3 — 0 

3xj f *2 b 3x3 = 11 

— 6xi—4x2 1-2x3= 30 

(c) 4xi — 8 x 2 = 12 

3xi — 6 x 2 — 9 

— 2xi -h 4x2 = — 6 

(d) 1 Oy — 4z 4 w = 1 

x I- 4y -z 4 w = 2 

3x 4 2^ 4 z \- 2w = 5 

— 2x — 8y | 2z — 2w = — 4 

x — 6y I 3z = 1 


Solve each of the systems in Exercise 8 by Gaussian elimination. 

9. 


Solve each of the following systems by Gauss-Jordan elimination. 

10 . 


(a) 5 xj — 2x2 b 6x3 = 0 
— 2 xi -h *2 + 3x3 — 1 


(b) x 1 — 2 x 2 b X3 — 4x4=1 
xi 4 3x2 4 7x34 2x4 = 2 
xi — 12x2 — 11*3 — 16 x 4 = 5 



(C) 


w + 2x -y = 4 
x -y = 3 
w 4 3x - 2.y = 7 
2u + 4v 4 w 4- 7x =7 


Solve each of the systems in Exercise 10 by Gaussian elimination. 

11 . 


Without using pencil and paper, determine which of the following homogeneous systems have nontrivial solutions. 

12 . 


(a) 2x\ — 3x2 I 4^3 — X 4 = 0 
7xi+ x 2 ~ 8*3 I 9x4 = 0 
2xi 4 8x2 I X3 — X4=0 

(b) xi + 3x2 _ *3 = 0 

X 2 — 8 x 3 = 0 
4 x 3 — 0 

(c) fliixi 4 (312X2 4 a 13x3 = 0 
(32ixi 4 (322X2 4 < 223*3 = 0 

(d) 3xi — 2x2 = 0 

6 x 1 — 4x2 = 0 


Solve the following homogeneous systems of linear equations by any method. 

(a) 2xi+ X 2 4 3x3 = 0 

xi + 2x2 = 0 

x 2 + x 3 = 0 

(b) 3xi + X 2 4 X 3 4 X 4 = 0 
5xi — X 2 4 X 3 — X 4 = 0 

(c) 2x f 2y I Az — 0 
w — y — 3z = 0 

2w 4 3x+ y 4 z = 0 
— 2w 4 x I- 3y> — 2z = 0 


14 . 


Solve the following homogeneous systems of linear equations by any method. 







(a) 2* — y — 3z = 0 
— 7 ; 4- 2 y — 3z = 0 

*4 y + 4z = 0 

(b) v I- 3w — 2?: = 0 
2& 4- v - 4w 4 3* = 0 
2u 4 3v 4- 2w — * = 0 

-4&s-3v I 5w-4x = 0 

(c) *1 4 3 x 2 I ^4 = 0 

*1 4 4*2 I 2*3 = 0 

— 2*2 — 2*3 —*4 = 0 
2 *i —4*2 4*3 I *4 = 0 
* 1 — 2*2 — *3- 4 * 4=0 


Solve the following systems by any method. 


(a) 2/i - 1 2 4 3/3 4 4 / 4 = 9 

1\ -2/34 7/4=11 

3/i -3/2 I I 3 4 5 / 4 = 8 
2/i 4 h 4 4/3 4 4/4= 10 

(b) Z 34 Z 4 fZ 5 = 0 
— Zi — Z 2 -4 2 Z 3 — 3 Z 4 -4 Zj 5 = 0 

Z 1 4 Z 2 — 2 Z 3 — Zj 5 = 0 

2Zi I 2 Z 2 — Z 3 4 Z^ = 0 


Solve the following systems, where a, b , and c are constants. 

16. 


(a) 2* I v = a 
3* I 6 y = b 


(b) *1 4 *2 + *3 = a 

2*i 4 2*3 = i 

3*2 4 3*3 = c 


For which values of a will the following system have no solutions? Exactly one solution? Infinitely many solutions? 

* I 2 y- 3z = 4 

3* — y I 5z = 2 

4* 4 y 4- (a 2 — 14)z = a 4 2 


17. 







Reduce 


18 . 


1 

■2 

4 


3 

29 

5 


to reduced row-echelon form. 


19. 


Find two different row-echelon forms of 

1 3' 

2 7 


Solve the following system of nonlinear equations for the unknown angles a, (3, and y, where 0 <a< 2x, 0 < {3 < 2« 

20. 0 < 7 < 7T. 

2 sin ct — cos ,3 4 3 tan ' — 3 
4 sinct 4 2 cos 3 — 2 tan ' = 2 
6 sin ct — 3 cos 3 I tan y = 9 


21 . 


Show that the following nonlinear system has 18 solutions if 0 < a < 2tt, 0 < 3 < 2ir, and 0 < 7 < 2jt. 

sinfi 4 2 cos ,f? -I- 3 tany = 0 
2 sin a + 5 cos 4 3 tan 7 = 0 
— sin— 5 cos ,f? 4 5 tan 7 — 0 


22 . 


For which value(s) of X, does the system of equations 
(A — 3)x 4- 7 = 0 

*4 (A-3)7 = 0 
have nontrivial solutions? 


23. 


24. 


Solve the system 

2x\ — X 2 = A*i 

2xi— *2 + *3 = ^2 
— 2xi 4 2x2 4*3= A*3 
for jq, X 2 , and x 3 in the two cases \ = A = 2- 

Solve the following system for x, y, and z. 


Id 1-1=1 

x y z 

I 4 - + - = 0 

x y z 

-I I -4 — = 5 
x 7 z 


Find the coefficients a, b, c, and d so that the curve shown in the accompanying figure is the graph of the equation 
25 - 7 =ax 3 I bx 2 4 cx 4 -d. 

Find coefficients a, b, c, and d so that the curve shown in the accompanying figure is given by the equation 
ax^ 4 < 37 ^ 4 bx 4 cy 4 d = 0 . 


•, and 


26 . 








27. 

(a) Show that if ad — be* 0- then the reduced row-echelon form of 

1 

0 

(b) Use part (a) to show that the system 

ax -\- by = k 
cx + dy = I 

has exactly one solution when ad-be* 0- 
Find an inconsistent linear system that has more unknowns than equations. 

28. 


0 

1 


a b 
c d 


Discussion 

Discovery 


Indicate all possible reduced row-echelon forms of 

29. 


(a) 


a b c 
d e f 

g h i 


(b) 


a 

s 

i 

m 


b 

f 

J 

n 


c d 

g k 

k 1 

P <i 


30 . 


Consider the system of equations 

















ax 4- by = 0 
cx dy = 0 
ex + fy = 0 

Discuss the relative positions of the lines ax -+ by = 0, cx 4- dy = 0, and g* + fy = 0 when (a) the 
system has only the trivial solution, and (b) the system has nontrivial solutions. 

Indicate whether the statement is always true or sometimes false. Justify your answer by giving a 
31* logical argument or a counterexample. 


(a) If a matrix is reduced to reduced row-echelon form by two different sequences of elementary 
row operations, the resulting matrices will be different. 

(b) If a matrix is reduced to row-echelon form by two different sequences of elementary row 
operations, the resulting matrices might be different. 

(c) If the reduced row-echelon form of the augmented matrix for a linear system has a row of 
zeros, then the system must have infinitely many solutions. 

(d) If three lines in the ^y-plane are sides of a triangle, then the system of equations formed from 
their equations has three solutions, one corresponding to each vertex. 


Indicate whether the statement is always true or sometimes false. Justify your answer by giving a 
32. logical argument or a counterexample. 


(a) A linear system of three equations in five unknowns must be consistent. 

(b) A linear system of five equations in three unknowns cannot be consistent. 

(c) If a linear system of n equations in n unknowns has n leading l's in the reduced row-echelon 
form of its augmented matrix, then the system has exactly one solution. 

(d) If a linear system of n equations in n unknowns has two equations that are multiples of one 
another, then the system is inconsistent. 


Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 



1.3 

MATRICES AND MATRIX 
OPERATIONS 


Rectangular arrays of real numbers arise in many contexts other than as 
augmented matrices for systems of linear equations. In this section we begin 
our study of matrix theory by giving some of the fundamental definitions of 
the subject. We shall see how matrices can be combined through the 
arithmetic operations of addition, subtraction, and multiplication. 


Matrix Notation and Terminology 

In Section 1.2 we used rectangular arrays of numbers, called augmented matrices, to abbreviate systems of linear equations. 
However, rectangular arrays of numbers occur in other contexts as well. For example, the following rectangular array with 
three rows and seven columns might describe the number of hours that a student spent studying three subjects during a 
certain week: 



Mon. 

Tues. 

Wed. 

Thurs. 

Fri. 

Sat. 

Sun. 

Math 

2 

3 

2 

4 

1 

4 

2 

History 

0 

3 

1 

4 

3 

2 

2 

Language 

4 

1 

3 

1 

0 

0 

2 


If we suppress the headings, then we are left with the following rectangular array of numbers with three rows and seven 
columns, called a “matrix”: 

'2 3 2 4 1 4 2' 

0 3 1 4 3 2 2 

4 13 10 0 2 

More generally, we make the following definition. 


DEFINITION 


A matrix is a rectangular array of numbers. The numbers in the array are called the entries in the matrix. 


Examples of Matrices 


Some examples of matrices are 


1 2' 


1 

1 

lol 

_I 


V 

_3_ 

3 0 

-1 4 

[2 1 0 -3], 

0 I 1 

0 0 0 



[ 4 ] 





















The size of a matrix is described in terms of the number of rows (horizontal lines) and columns (vertical lines) it contains. 
For example, the first matrix in Example 1 has three rows and two columns, so its size is 3 by 2 (written 3 x 2)- In a size 
description, the first number always denotes the number of rows, and the second denotes the number of columns. The 
remaining matrices in Example 1 have sizes 1x4, 3x3, 2x1, and lxl, respectively. A matrix with only one column is 
called a column matrix (or a column vector), and a matrix with only one row is called a row matrix (or a row vector ). Thus, 
in Example 1 the 2 x 1 matrix is a column matrix, the 1 x 4 matrix is a row matrix, and the 1 x 1 matrix is both a row matrix 
and a column matrix. (The term vector has another meaning that we will discuss in subsequent chapters.) 


It is common practice to omit the brackets from a 1 x 1 matrix. Thus we might write 4 rather than [4], Although 
this makes it impossible to tell whether 4 denotes the number “four” or the lxl matrix whose entry is “four,” this rarely 
causes problems, since it is usually possible to tell which is meant from the context in which the symbol appears. 


We shall use capital letters to denote matrices and lowercase letters to denote numerical quantities; thus we might write 


A = 


2 1 7 

3 4 2 


or 


C = 


a b 
d e 


f 


When discussing matrices, it is common to refer to numerical quantities as scalars. Unless stated otherwise, scalars will be 
real numbers', complex scalars will be considered in Chapter 10. 


The entry that occurs in row i and column j of a matrix A will be denoted by Thus a general 3x4 matrix might be written 
as 



a 11 

a \2 

3 13 

«14 

A = 

321 

322 

« 23 

«24 


331 

332 

a 33 

334 


and a general m x n matrix as 



311 

312 ‘ 

" a ln 

321 

322 ■ 


3jjj1 

3 m2 L 



( 1 ) 


When compactness of notation is desired, the preceding matrix can be written as 

to?]*™ or K?] 

the first notation being used when it is important in the discussion to know the size, and the second being used when the size 
need not be emphasized. Usually, we shall match the letter denoting a matrix with the letter denoting its entries; thus, for a 
matrix B we would generally use b i} for the entry in row i and column j, and for a matrix C we would use the notation Cjj. 


The entry in row i and column j of a matrix A is also commonly denoted by the symbol (A) ij- Thus, for matrix 1 above, we 
have 


and for the matrix 


(^)y ~ a ij 



we have (^4) j j = 2, (j4) 12 = — 3, (-^4) 21 = 7’ 4nd (^ 4)22 = 0* 


Row and column matrices are of special importance, and it is common practice to denote them by boldface lowercase letters 
rather than capital letters. For such matrices, double subscripting of the entries is unnecessary. Thus a general 1 x n row 
matrix a and a general m x 1 column matrix b would be written as 












a=[fli A2 ■“ a n] and 1) 


*1 

h 

bm 

A matrix A with n rows and n columns is called a square matrix of order it, and the shaded entries a\\, a 22’ •••> in 2 are 
said to be on the main diagonal of A. 


an 

ai2 ■■ 

" a ln 

fl 21 

ct'12 •' 


a n 1 

a rQ. " 



Operations on Matrices 

So far, we have used matrices to abbreviate the work in solving systems of linear equations. For other applications, however, 
it is desirable to develop an “arithmetic of matrices” in which matrices can be added, subtracted, and multiplied in a useful 
way. The remainder of this section will be devoted to developing this arithmetic. 


DEFINITION 


Two matrices are defined to be equal if they have the same size and their corresponding entries are equal. 


In matrix notation, if A = [a^ ] and B = [&,y] have the same size, then A —B if and only if (A) ^ = (B) or, equivalently, 
a : j — bjj for all i and j. 


Equality of Matrices 


Consider the matrices 


A = 


2 1 
3 x 


2 

3 5 


2 1 0 
3 4 0 


If x = 5, then A = B, but for all other values of x the matrices A and B are not equal, since not all of their corresponding 
entries are equal. There is no value of x for which A = C since A and C have different sizes. 


DEFINITION 


If A and B are matrices of the same size, then the sum A 4 B is the matrix obtained by adding the entries of B to the 
corresponding entries of A, and the difference A — B is the matrix obtained by subtracting the entries of B from the 
corresponding entries of A. Matrices of different sizes cannot be added or subtracted. 


In matrix notation, if A = [a,j ] and B = [£j,y] have the same size, then 
























(A ! B) ij — (A) ij 4- (3) ; w — a-ij I 


and (A — B ) y — (-4) — (5) ^ 


Addition and Subtraction 


Consider the matrices 



'2 10 3' 


'-4 3 5 1' 


'1 f 
_2 2_ 

A = 

-1 0 2 4 

4-270 

, 5 = 

2 2 0 -1 

3 2-4 5 

c = 


Then 



-2 4 5 4 


6_2-5 2 

A f 5 = 

12 2 3 

7 0 3 5 

and A — B — 

-3-2 2 5 

1 -4 11 -5 


The expressions A 4- C, B \- C, A — C, an d B — C are undefined. 


♦ 


DEFINITION 


If A is any matrix and c is any scalar, then the product C A is the matrix obtained by multiplying each entry of the matrix A 
by c. The matrix C A is said to be a scalar multiple of A. 


In matrix notation, if A — ], then 


(cA) jj — c (A) jj — c&ij 


Scalar Multiples 


For the matrices 


we have 


A = 


2 3 4 
1 3 1 


B = 


0 2 7 

-13 -5 


9-6 3 

3 0 12 


4 6 
2 6 2 


(-1)5 = 


2 

-7' 

l c - 

'3 

-2 r 

3 

5 

3 C ~ 

1 

—1 

^r 

0 


It is common practice to denote ( _ \)B by _ B- 


♦ 


If A\, A' 2 ' •••■> A n are matrices of the same size and c\, ci, ..., c u are scalars, then an expression of the form 

c\A\ + C 2 A 2 H-1- c„A„ 

is called a linear combination of 2 , ..., A n with coefficients c\, cj, ■ ■c n . For example, if A, B, and C are the matrices 
in Example 4, then 

































2A-B + ±C = 2A (- 1)5 4 


4 

6 

8" 

+ 

'0 

-2 

-7' 

+ 

1 

u.J 

1 

[\j 

_1 

= 

'7 

2 

2" 

2 

6 

2 

1 

-3 

5 


1 

O 

1 _ 


4 

3 

11 


is the linear combination of A, B, and C with scalar coefficients 2,-1, and -j. 


Thus far we have defined multiplication of a matrix by a scalar but not the multiplication of two matrices. Since matrices are 
added by adding corresponding entries and subtracted by subtracting corresponding entries, it would seem natural to define 
multiplication of matrices by multiplying corresponding entries. However, it turns out that such a definition would not be 
very useful for most problems. Experience has led mathematicians to the following more useful definition of matrix 
multiplication. 


DEFINITION 


If A is an m x r matrix and B is an matrix, then the product AB is the mxn matrix whose entries are determined as 
follows. To find the entry in row i and column j of AB, single out row i from the matrix A and column j from the matrix B. 
Multiply the corresponding entries from the row and column together, and then add up the resulting products. 


Multiplying Matrices 


Consider the matrices 

4 14 3' 

0-131 
2 7 5 2 

Since A is a 2 x 3 matrix and B is a 3 x 4 matrix, the product AB is a 2 x 4 matrix. To determine, for example, the entry in 
row 2 and column 3 of AB, we single out row 2 from A and column 3 from B. Then, as illustrated below, we multiply 
corresponding entries together and add up these products. 


A = 





'1 2 

4" 

II 

2 6 

0_ 





1 

2 


2 4 
6 0 


4 1 4 
0-13 
[2 7 5 

(2-4) + (6- 


3 
1 

2 

3) + (0-5) = 26 


□ 



□ 


□ 


26 

□ 


The entry in row 1 and column 4 of AB is computed as follows: 


1 2 4 

2 6 0 


4 1 4 

0-13 
_2 7 5 

(1 - 3) + (2 - 


□ 



LoJ 


□ 



□ 


1) + (4-2) = 13 


The computations for the remaining entries are 



































AB = 


(1-4) A (2-0) 

+ (4-2) = 

12 


(1.1) _ (2 -1) 

+ (4 - 7) = 

27 


(1-4) + (2-3) 

+ (4-5) = 

30 

8 

'12 

(2-4) +(6-0) 

+ (0 - 2) = 

_ 3 

(2 -1) — (6 -1) 

+ (0 - 7) = 

-4 


(2 - 3) + (6 -1) 

+ (0 - 2) = 

12 



27 30 13 
-4 26 12 


♦ 


The definition of matrix multiplication requires that the number of columns of the first factor A be the same as the number of 
rows of the second factor B in order to form the product AB • If this condition is not satisfied, the product is undefined. A 
convenient way to determine whether a product of two matrices is defined is to write down the size of the first factor and, to 
the right of it, write down the size of the second factor. If, as in 3, the inside numbers are the same, then the product is 
defined. The outside numbers then give the size of the product. 


4 B AB 

m x r r x n = m x tt 

Inside * 

Outs iite 


(3) 


Determining Whether a Product Is Defined 


Suppose that A, B, and C are matrices with the following sizes: 

ABC 
3x4 4x7 7x3 

Then by 3, AB is defined and is a 3 x 7 matrix; BC is defined and is a 4 x 3 matrix; and CA is defined and is a 7 x 4 matrix. 
The products AC, CB, and BA are a ll undefined. 

* 


In general, if A — [a^ ] is an m x r matrix and B — [&,y] is an rxn matrix, then, as illustrated by the shading in 4, 


'an 

«21 

a \2 ■ 

«22 ‘ 

'■ «1 r~ 
- «2 r 

*11 

*12 ■ 

- iy ■ 

" ^1h 

; 



*21 

*22 ■ 

■ • b 2 j - 


an 

a i2 ■ 

- air 

: 

: 


: 


a m2 ‘ 

“ a mr 

*n 

*r2 ‘ 

'■ b rj ■ 

-1 

£ 


the entry (AB) ij in row i and column j of AB is given by 

(AB)ij - anbij A a i2 b 2 j A a l3 b 3] A - + a ir b r] 


(4) 


(5) 


Partitioned Matrices 

A matrix can be subdivided or partitioned into smaller matrices by inserting horizontal and vertical rules between selected 
rows and columns. For example, the following are three possible partitions of a general 3 x 4 matrix A—the first is a partition 
of A into four submatrices A\ 2 , A 2 \, an d A 22 , th e second is a partition of A into its row matrices iq, y 2 , and 1 - 3 ; and the 
third is a partition of A into its column matrices <q, t 2 , C 3 , a nd 04 : 














A = 

Oil 

0)2 

On 

014 

■ 

[A,, 

A12 

ct 2\ 

if 22 

033 


" 

I_A 3 i 

A22J 


031 

032 

«33 

034 

. 


Oil 

it] 2 

on 

014 


n 


A = 

a 2\ 

a 22 

#23 

024 

— 

ri 



031 


033 

034 


T3 



on 

0)2 

0(3 

014 




A = 

021 


033 

034 

= |ci 

Ci C3 


031 

032 

033 

034 





Matrix Multiplication by Columns and by Rows 

Sometimes it may be desirable to find a particular row or column of a matrix product AB without computing the entire 
product. The following results, whose proofs are left as exercises, are useful for that purpose: 


yth column matrix of AB = A [yth column matrix of B] 


( 6 ) 


zth row matrix of AB = [ zth row matrix of A] B 


(7) 


Example 5 Revisited 


If A and B are the matrices in Example 5, then from 6 the second column matrix of AB can be obtained by the computation 


'1 2 4" 
2 6 0_ 

r 

- 1 

= 

27' 
_ -4_ 


7 

T 


T 


Second column Second column 
of B of AB 


and from 7 the first row matrix of AB can be obtained by the computation 

[4 14 3 

First row of A —> [ 1 2 4]' 


0 - 1 
2 7 


= [12 27 30 13] «- First row of AB 


♦ 


If ap ^ 2 , • ••, denote the row matrices of A and bp b 2 > •••> b M denote the column matrices of B, then it follows from 

Formulas 6 and 7 that 


AB = A[ bi b 2 - b„] = [Abi Ab 2 - Ah n ] 

[AB computed column by column) 


( 8 ) 




























(AB computed row by row 7 ) 


Remark Formulas 8 and 9 are special cases of a more general procedure for multiplying partitioned matrices (see Exercises 
15, 16 and 17). 

Matrix Products as Linear Combinations 


Row and column matrices provide an alternative way of thinking about matrix multiplication. For example, suppose that 


Then 


Ax = 


'ail 

<*12 ■ 




"*r 

321 

«22 ■ 


and 

x = 

*2 

a ml 

a m2 L 




*h 


"<311*1 + <312*2 + “ + <31w*h 


"<311 ' 


"<312 " 



^ 1h 

<321*1 +<*22*2 + “ + <32h*h 

= *1 

<321 

+ *2 

<322 

+ - 

" + *H 

a 2 H 

<3ffll*l + <3m2*2 ”1 f <3 j?jm*h 


<3jnl 


<3 m2 



a 7nn 


( 10 ) 


In words, 10 tells us that the product °f a matrix A with a column matrix x is a linear combination of the column matrices 
of A with the coefficients coming from the matrix x. In the exercises we ask the reader to show that the product yA of a \ xm 
matrix y with an m x n matrix A is a linear combination of the row matrices of A with scalar coefficients coming from y. 


Linear Combinations 


The matrix product 


'-13 2' 

2' 


f 

12-3 

- 1 

= 

-9 

2 1-2 

3 


-3 


can be written as the linear combination of column matrices 


- 1 


3 


2 


1 

1 

- 1 

2 

+ 3 

-3 

= 

-9 

2 


1 


-2 


-3 


The matrix product 


[1 -9 -3] 


-1 3 
1 2 
2 1 



= [-16 


- 18 35] 


can be written as the linear combination of row matrices 


1[-1 3 2] — 9[ 1 2 —3] — 3[2 1 — 2] = [ — 16 - 13 35] 


♦ 


It follows from 8 and 10 that the jth column matrix of a product AB is a linear combination of the column matrices of A with 
the coefficients coming from the jth column ofB. 



































Columns of a Product AB as Linear Combinations 


We showed in Example 5 that 

12 27 30 13" 

3 -4 26 12_ 

The column matrices of AB can be expressed as linear combinations of the column matrices of A as follows: 

"12 
8 

27 
-4 

"30 
26 

"13 
12 

♦ 


= 4 


+ 0 


+ 2 


+ 7 


= 4 


+ 3 


+ 5 


= 3 


+ 


+ 2 


AB = 


1 2 4 

2 6 0 


4 14 3 
0-131 
2 7 5 2 


Matrix Form of a Linear System 

Matrix multiplication has an important application to systems of linear equations. Consider any system of m linear equations 
in n u n knowns. 


311*1 T 312*2 + ■*■+ ai„x„ = b\ 
321*1 + 322*2 + ■'■+ 32m*h = *2 


3ml*l 4- 3 m 2*2 4 b 3 jjjh*m — 

Since two matrices are equal if and only if their corresponding entries are equal, we can replace the m equations in this 
system by the single matrix equation 


311*1 d 312*2 +■ 

" “I" 


*1 

321*1 A 322* 2 +■ 

" “I" 

= 

H 

3ml* 1 4 3 jjj2*2 A ■ 

,J 1 




The m x 1 matrix on the left side of this equation can be written as a product to give 


'311 312 ■ 

321 322 ■ 

" n 

,J a 2n 


i 

t-h r-1 

I_ 


_ *r 

h 

3ml 3 m 2 ‘ 

,J 


*M 




If we designate these matrices by A, x, and b, respectively, then the original system of m equations in n unknowns has been 
replaced by the single matrix equation 

Ax = b 

The matrix A in this equation is called the coefficient matrix of the system. The augmented matrix for the system is obtained 
by adjoining b to A as the last column; thus the augmented matrix is 





























































ail 

a 12 

a 1h 


[A|b] = 

<*21 

«22 


h 


a m\ 

a m2 


bm 


Matrices Defining Functions 

The equation Ax = b with A and b given defines a linear system to be solved for x. But we could also write this equation as 
y = Ax, where A and x are given. In this case, we want to compute y. If A is m x «, then this is a function that associates with 
every n x 1 column vector x an 1 column vector y, and we may view A as defining a rule that shows how a given x is 
mapped into a corresponding y. This idea is discussed in more detail starting in Section 4.2. 


EXAI\ A Function Using Matrices 


Consider the following matrices. 



The product y — Ax is 


'1 

o' 

“a“ 


a“ 

_0 

-1_ 

b_ 


_-b_ 


so the effect of multiplying A by a column vector is to change the sign of the second entry of the column vector. For the 
matrix 


B = 


0 1 

-1 0 


the product y = Bx is 


0 

f 

“a“ 


b 

_ - 1 

0 _ 

b_ 


— a 


so the effect of multiplying B by a column vector is to interchange the first and second entries of the column vector, also 
changing the sign of the first entry. 


If we view the column vector x as locating a point (a, b) in the plane, then the effect of A is to reflect the point about the 
x-axis (Figure 1.3.1a) whereas the effect of B is to rotate the line segment from the origin to the point through a right angle 
(Figure 1.3.1(f). 

























y 


-h - * y 


<*ri 



Figure 1.3.1 


♦ 


Transpose of a Matrix 

We conclude this section by defining two matrix operations that have no analogs in the real numbers. 


DEFINITION 


If A is any m x n matrix, then the transpose of A, denoted by is defined to be the n x m matrix that results from 
interchanging the rows and columns of A; that is, the first column of A T is the first row of A, the second column of A T is 
the second row of A, and so forth. 


Some Transposes 


The following are some examples of matrices and their transposes. 


a 11 

a n 

a 13 

a 14' 



'2 

3' 

<*21 

a 22 

a 23 

&24 

B = 

1 

4 

fl 31 

a 32 

a 33 

fl 34 



5 

6 

a 11 

«21 

a 31" 






a n 

a 22 

a 32 


b 7 = 

'2 

1 

5' 

« 13 

&22 

a 33 


_3 

4 

6_ 

a \A 

a 24 

a 34 







C=[l 3 5], D=[ 4] 



D T = 


[4] 


♦ 


Observe that not only are the columns of A T the rows of A, but the rows of A T are the columns of A. Thus the entry in row i 























and column j of A T is the entry in row j and column i of/4; that is, 


04% = 0% 


( 11 ) 


Note the reversal of the subscripts. 

In the special case where A is a square matrix, the transpose of A can be obtained by interchanging entries that are 
symmetrically positioned about the main diagonal. In 12 it is shown that /J can also be obtained by “reflecting” A about its 
main diagonal. 



1 

to 

-t- 

i 


(4) 


I 3 -5" 

A = 

3 7 D 

-5 8 6 


S) (§%"*. 

i 

a t = 

1 

30 VC 

r- 0 

(N 

1 

_ 1 


1 


Interchange ciiincs ihij« ore 
>v in i n ctrica I \y jnio ned 
Jihmx rlji. mam diagjpiuiL 


( 12 ) 


DEFINITION 


If A is a square matrix, then the trace of A, denoted by tr(A), is defined to be the sum of the entries on the main diagonal 
of A. The trace of A is undefined if A is not a square matrix. 


EXAMPLE 12 Trace of a Matrix 


The following are examples of matrices and their traces. 

- 1 2 7 0 " 

3 5-84 
1 2 7-3 

4-210 

tr(A) = a\i + aji + ^33 tr(i?) — —1 + 5 + 7 + 0 = 11 


A = 


a 11 ^12 fli3 
a 2\ a 22 a 22 
<331 a 22 «33 


B = 


Exercise Set 1.3 



Click here for Just Ask! 


Suppose that A, B, C, D, and E are matrices with the following sizes: 

1 . 





















A 

(4x5) 


B 

(4x5) 


C D E 

(5x2) (4x2) (5x4) 


Determine which of the following matrix expressions are defined. For those that are defined, give the size of the resulting 
matrix. 


(a) BA 

(b) AC 4 D 

(c) AE I B 

(d) AB + B 

(e) E(A F B) 

(f) E(AC) 

(g) E t A 

(h) (. A r + E)D 


Solve the following matrix equation for a, b, c, and d. 

2 . 


a—b b 1 c 


'8 f 

3d F c 2a —Ad 


1 6_ 


3. 


Consider the matrices 


A = 


3 0 
-1 2 
1 1 



C = 


1 4 2 
3 1 5 



'15 2" 


"6 13" 

D = 

-1 0 1 

. E = 

-1 1 2 


3 2 4 


1 

r-~i 

_I 


Compute the following (where possible). 


(a) r>4 R 



(b) D-E 


(c) 5,4 

(d) -7C 

(e) 2 B-C 

(f) AE — 2D 

(g) — 3{D\ 2 E) 

(h) A — A 

(i) tr(£>) 

0) tr(D-3E) 

(k) 4 tr(75) 

(l) ir(A) 

Using the matrices in Exercise 3, compute the following (where possible). 

(a) 2 A t +C 

(b) D T -E T 

(c) (j D-E) 7 

(d) b T + 5C T 

(e) ±C r -±A 

(f) B-B t 



(g) 2E t -3D t 


Using the matrices in Exercise 3, compute the following (where possible). 

5. 

(a) AS 

(b) BA 

(c) (3 E)D 

(d) (AB) C 

(e) A(BC) 

(f) CC T 

(g) (DA) 1 

(h) (C T B)A T 

(i) tr (DD T ) 

G) tr(4 E T -D) 

(k) tr( C T A T \-2E r ) 

Using the matrices in Exercise 3, compute the following (where possible). 

(a) (2D T -E).A 

(b) (4B)C+2B 


6 . 



(C) ( — AC) T 4 5D t 


(d) (BA t -2C) T 

(e) B T (CC T -A T A) 

(f) D T E T -{ED) T 


'3 

-2 

7' 


'6 

-2 

4' 

6 

5 

4 

and B = 

0 

1 

3 

0 

4 

9 


7 

7 

5 


Use the method of Example 7 to find 

(a) the first row of AB 


(b) the third row of AB 


(c) the second column of AB 


(d) the first column of BA 


(e) the third row of AA 


(f) the third column of AA 


Let A and B be the matrices in Exercise 7. Use the method of Example 9 to 

8 . 

(a) express each column matrix of AB as a linear combination of the column matrices of A 

(b) express each column matrix of BA as a linear combination of the column matrices of B 


9 . 


Let 


a\\ a 12 - tsfi„ 
321 «22 - < 32 n 

a ml a m2 ■" 


y=[y\ y2 ■■■ y m ] and A = 



(a) Show that the product vj 4 can be expressed as a linear combination of the row matrices of A with the scalar 
coefficients coming frorny. 


(b) Relate this to the method of Example 8. 


Hint Use the transpose operation. 

Let A and B be the matrices in Exercise 7. 

10 . 


(a) Use the result in Exercise 9 to express each row matrix of AB as a linear combination of the row matrices of B. 


(b) Use the result in Exercise 9 to express each row matrix of BA as a linear combination of the row matrices of A. 


Let C, D, and E be the matrices in Exercise 3. Using as few computations as possible, determine the entry in row 2 and 
11 * column 3 of C(DE)- 


12 . 

(a) Show that if AB and BA are both defined, then AB and BA are square matrices. 


(b) Show that if A is an m x n matrix and A{BA) is defined, then B is an w x m matrix. 


In each part, find matrices A, x, and b that express the given system of linear equations as a single matrix equation 
13* j4x = 1). 

(a) 2^1 — 3x2 4 5x3= 7 

9 xi - X2 I X3 = — 1 

xi I 5x2 I 4x3= 0 

(b) 4xi -3x3-1- *4=1 

5xi + X2 — 3x4 = 3 
2xi — 5x 2 4 9x3— *4 = 0 

3x2 — *3 I 7*4 = 2 


14 . 


In each part, express the matrix equation as a system of linear equations. 




(a) 



If A and B are partitioned into submatrices, for example, 



then can be expressed as 


A 11 H} i + A 

A ii B12 + A12B22 

"1] jfl] | H~ A y y /? > | 

U 

1 B 13 4 - A 2 ? B22 


provided the sizes of the submatrices of A and B are such that the indicated operations can be performed. This method of 
multiplying partitioned matrices is called block multiplication. In each part, compute the product by block 
multiplication. Check your results by multiplying directly. 



(b) 







(c) 


2 -1 

3 

-4 

0 1 

5 

7 


1 4 


1 £) 0 

0 O' 


■ 3 3" 

0 1 0 

0 0 


-1 4 

0 0 1 

0 0 


1 5 

0 £) 0 

2 0 


2 -2 

,0 0 0 

-1 2j 


. 1 6 _ 


In each part, determine whether block multiplication can be used to compute AB from the given partitions. If so, compute 
17. the product by block multiplication. 

Note See Exercise 15. 


(a) 


(b) 



18 . 

(a) Show that if A has a row of zeros and B is any matrix for which AB is defined, then AB also has a row of zeros. 

(b) Find a similar result involving a column of zeros. 

Let A be any mxn matrix and let 0 be the mxn matrix each of whose entries is zero. Show that if kA = then k = ij or 
19 - A = 0- 

Let I be the matrix whose entry in row i and column j is 

20. / H . 

|1 li I=j 

| 0 if i±j 

Show that AI = lA = A for every n x.n matrix A. 


21 . 


In each part, find a 6 x 6 matrix [a^] that satisfies the stated condition. Make your answers as general as possible by 
using letters rather than specific numbers for the nonzero entries. 





(a) Ajy = 0 if i j 

(b) a-ij = 0 if i > j 

(c) fljj = 0 if j < j 

(d) j — 0 if |z — j I > 1 

Find the 4x4 matrix j 4 = [a,-,-] whose entries satisfy the stated condition. 

22 . 

(a) a,jf = z + y 

(b) 

( c ) ( 1 if |z — y| > 1 

flv = |-i if 


Consider the function y — f (x ) defined for 2 x 1 matrices x by y— Ax, where 


Plot f(x) together with x in each case below. How would you describe the action off! 



(d) 


x = 


2 

-2 



Let Abea«xfM matrix. Show that if the function y = f (x) defined for m x 1 matrices x by y = Ax satisfies the linearity 
24* property, then j (aw 4- j3z) — af (w) 4- 3f (z) for any real numbers a and (3 and any m 1 matrices w and z. 

Prove: If A and B are « x n matrices, then tr (A -I- B) — tr (A) 4 tr(£?) ■ 


Discussion 

Discovery 


Describe three different methods for computing a matrix product, and illustrate the methods by 
26. computing some product AS three different ways. 


How many 3x3 matrices A can you find such that 

x Ay' 
x-y 
o 

for all choices of x, y, and z? 



28. 


How many 3x3 matrices A can you find such that 


~x " 


~*y~ 


= 

0 

z 


0 


for all choices of x, y, and z? 


A matrix B is said to be a square root of a matrix A if BB = A- 

29. 


(a) 


Find two square roots of A = 


2 

2 


2 

2 


(b) 


How many different square roots can you find of A = 


5 

0 


0 

9 


? 


(c) Do you think that every 2x2 matrix has at least one square root? Explain your 
reasoning. 


Let 0 denote a 2 x 2 matrix, each of whose entries is zero. 

30. 


(a) Is there a 2 x 2 matrix A such that A ^ 0 and AA = 0? Justify your answer. 

(b) Is there a 2 x 2 matrix A such that A * 0 and AA = A? Justify your answer. 


31 . 


Indicate whether the statement is always true or sometimes false. Justify your answer with a 
logical argument or a counterexample. 
















(a) The expressions tr (AA T ) and tr(A T A) are always defined, regardless of the size of A. 


(b) tr {AA 7 ') = tr(A 7 A) for every matrix A. 


(c) If the first column of A has all zeros, then so does the first column of every product AB- 

(d) If the first row of A has all zeros, then so does the first row of every product AB- 


Indicate whether the statement is always true or sometimes false. Justify your answer with a 
32. logical argument or a counterexample. 


(a) If A is a square matrix with two identical rows, then AA has two identical rows. 

(b) If A is a square matrix and AA has a column of zeros, then A must have a column of 
zeros. 


(c) If B is an n x n matrix whose entries are positive even integers, and if A is an n x n 
matrix whose entries are positive integers, then the entries of AB and BA are positive 
even integers. 

(d) If the matrix sum AB 4 BA is defined, then A and B must be square. 


Suppose the array 

'4 3 3' 

2 1 0 

4 4 2 

represents the orders placed by three individuals at a fast-food restaurant. The first person 
orders 4 burgers, 3 sodas, and 3 fries; the second orders 2 burgers and 1 soda, and the third 
orders 4 burgers, 4 sodas, and 2 fries. Burgers cost $2 each, sodas $1 each, and fries $1.50 
each. 


(a) Argue that the amounts owed by these persons may be represented as a function 
y — f (*), where f (*) is equal to the array given above times a certain vector. 


(b) Compute the amounts owed in this case by performing the appropriate multiplication. 




(c) Change the matrix for the case in which the second person orders an additional soda 
and 2 fries, and recompute the costs. 
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1.4 

INVERSES; RULES OF 
MATRIX ARITHMETIC 


In this section we shall discuss some properties of the arithmetic operations 
on matrices. We shall see that many of the basic rules of arithmetic for real 
numbers also hold for matrices, but a few do not. 


Properties of Matrix Operations 

For real numbers a and b, we always have a b = ba, which is called the commutative law for multiplication. For matrices, 
however, AB and BA need not be equal. Equality can fail to hold for three reasons: It can happen that the product AB is defined 
but BA is undefined. For example, this is the case if A is a 2 x 3 matrix and B is a 3 x 4 matrix. Also, it can happen that AB and 
BA are both defined but have different sizes. This is the situation if A is a 2 x 3 matrix and B is a 3 x 2 matrix. Finally, as 
Example 1 shows, it is possible to have AB * BA even if both AB and BA are defined and have the same size. 


AB and BA Need Not Be Equal 


Consider the matrices 


Multiplying gives 




2 

0 



BA = 


3 6 
-3 0 


Thus, AB a BA- 

♦ 

Although the commutative law for multiplication is not valid in matrix arithmetic, many familiar laws of arithmetic are valid 
for matrices. Some of the most important ones and their names are summarized in the following theorem. 


THEOREM 1.4.1 


Properties of Matrix Arithmetic 

Assuming that the sizes of the matrices are such that the indicated operations can be performed, the following rules of 
matrix arithmetic are valid. 

(a) A V B — B -\- A (Commutative law for addition) 

(b) A-h (B A- C) = (A + B) 4- C (Associative law for addition) 

(c) A(BC') — {AB')C (Associative law for multiplication) 















(d) 

A(B + C) 

= AB + AC 

(Left distributive law) 

(e) 

(B + C)A 

= BA + CA 

(Right distiibutive law) 

(f) 

A(B-C) 

= AB —AC 


(g) 

(B - C)A 

= BA-CA 


(h) 

a(B+C) 

= aB V aC 


(i) 

a(B-C) 

— aB — aC 


0) 

(a A b)C 

= aC + bC 


(k) 

(a-b)C-- 

— aC — bC 


(1) 

a{bC) = ( 

[ab)C 


(m) 

a(BC) = i 

{aB)C = B{aC) 



To prove the equalities in this theorem, we must show that the matrix on the left side has the same size as the matrix on the 
right side and that corresponding entries on the two sides are equal. With the exception of the associative law in part (c), the 
proofs all follow the same general pattern. We shall prove part (d) as an illustration. The proof of the associative law, which is 
more complicated, is outlined in the exercises. 


Proof (d) We must show that A{B V- C) and AB + AC have the same size and that corresponding entries are equal. To form 
A(B 4- C), the matrices B and C must have the same size, say mxn, and the matrix A must then have m columns, so its size 
must be of the form r x m- This makes A(B 4 C") an rxn matrix. It follows that AB I AC is also an rxn matrix and, 
consequently, A{B + C) and AB 4 AC have the same size. 

Suppose that A = [ay ], B = [£y ], and C = [cy ]. We want to show that corresponding entries of A(B 4 C) and AB |- AC are 
equal; that is, 

[A(8 + 0)] if =[AB 4 JiC] if 

for all values of i and j. But from the definitions of matrix addition and matrix multiplication, we have 

[A{B 4 C) ] y = a,i (b y 4 cy) 4 2j f c 2j ) ^ -f + c mj) 

= (flubij 4 a i2 b 2 j I-1 + (flycy 4 a i2 c 2 j 4 -4 

= [AB] iJ +[AC] iJ =[AB + AC] ii 



Remark Although the operations of matrix addition and matrix multiplication were defined for pairs of matrices, associative 
laws ( b ) and (c) enable us to denote sums and products of three matrices as A 4- B 4- C and ABC without inserting any 
parentheses. This is justified by the fact that no matter how parentheses are inserted, the associative laws guarantee that the 
same end result will be obtained. In general, given any sum or any product of matrices, pairs of parentheses can be inserted or 
deleted anywhere within the expression without affecting the end result. 


EXAMPLE 2 Associativity of Matrix Multiplication 


As an illustration of the associative law for matrix multiplication, consider 

i 1 2 

A= ' 


3 4 
0 1 


B = 


4 3 
2 1 


C = 


1 0 
2 3 


Then 


AB = 


'1 2' 



CO 

3 4 

0 1 

4 3 

2 1_ 

= 

20 13 

2 1 


and BC = 


Thus 


{AB)C = 

' 8 
20 

5' 

13 

'1 O' 
2 3 

= 

'18 

46 

15' 

39 


2 

1 



4 

3 


and 

A{BC) = 

so (AB)C = A(BC), as guaranteed by Theorem 1.4.1c. 


'1 2' 
3 4 

'10 9' 
4 3 

= 

'18 15' 
46 39 

0 1 



4 3 


'4 3' 

- 1 

O 


'10 9' 

_2 1_ 

2 3 _ 


- 1 

_1 


Zero Matrices 

A matrix, all of whose entries are zero, such as 


0 0 
0 0 


0 0 0 
0 0 0 
0 0 0 


0 0 0 0 
0 0 0 0 


[ 0 ] 


is called a zero matrix. A zero matrix will be denoted by 0; if it is important to emphasize the size, we shall write 0 mxyi for the 


m x n 

zero matrix. Moreover, in keeping with our convention of using boldface symbols for matrices with one column, we will 
denote a zero matrix with one column by 0 . 

If A is any matrix and 0 is the zero matrix with the same size, it is obvious that A A 0 = 0 A A = A- The matrix 0 plays much 
the same role in these matrix equations as the number 0 plays in the numerical equations a -\- = Q -\- a = a- 


Since we already know that some of the rules of arithmetic for real numbers do not carry over to matrix arithmetic, it would be 
foolhardy to assume that all the properties of the real number zero carry over to zero matrices. For example, consider the 
following two standard results in the arithmetic of real numbers. 










































If ab — ac and a * 0> then b-c- (This is called the cancellation law.) 

If ad = 0> then at least one of the factors on the left is 0. 


As the next example shows, the corresponding results are not generally true in matrix arithmetic. 


The Cancellation Law Does Not Hold 


Consider the matrices 


You should verify that 


A = 


'o r 
.0 2 _ 

ii 

'1 f 
_3 4_ 

II 

O 

l l 

Uj DO 

U-I 

1_1 

. D = 

I I 

O U.j 

O ---j 

1_1 


AB —AC — 


2 

6 


4 

8 


and 



0 

0 


Thus, although A±0, it is incorrect to cancel the A from both sides of the equation AB = AC and write B = C- Also, AD = C, 
yet A±0 and D^O- Thus, the cancellation law is not valid for matrix multiplication, and it is possible for a product of matrices 
to be zero without either factor being zero. 

♦ 


In spite of the above example, there are a number of familiar properties of the real number 0 that do carry over to zero matrices. 
Some of the more important ones are summarized in the next theorem. The proofs are left as exercises. 


THEOREM 1.4.2 


Properties of Zero Matrices 

Assuming that the sizes of the matrices are such that the indicated operations can be performed, the following rules of 
matrix arithmetic are valid. 


(a) A + 0 = 0 4 A = A 

(b) A-A = 0 

(c) 0-A= -A 

(d) AO = 0: 0A=0 


Identity Matrices 

Of special interest are square matrices with l's on the main diagonal and 0's off the main diagonal, such as 

















and so on. 


1 0 
0 1 


1 0 0 
0 1 0 
0 0 1 


10 0 0 
0 10 0 
0 0 10 
0 0 0 1 


A matrix of this form is called an identity matrix and is denoted by 1. If it is important to emphasize the size, we shall write / 
for the n x n identity matrix. 


If A is an m x n matrix, then, as illustrated in the next example, 

AI n — A and I m A — A 

Thus, an identity matrix plays much the same role in matrix arithmetic that the number 1 plays in the numerical relationships 
a -1 = 1 - a = a- 


Multiplication by an Identity Matrix 


Consider the matrix 


Then 


and 


an a 12 a i 3 
a 2\ a 22 a 22 


1 2 A = 


'1 

o' 

■fill 

an 

a\2 


'an 

a \2 

a\ 3 ~ 

_0 

1_ 

fl 21 

a 22 

a 23 


a 21 

a 22 

a 23 


= A 


AI 3 = 


a 11 
a 2\ 


a 12 a i3 

a 2 2 ajl 


1 0 0 
0 1 0 
0 0 1 


a li 
fl 21 


a 12 a i3 

a 2 2 ajl 


♦ 


As the next theorem shows, identity matrices arise naturally in studying reduced row-echelon forms of square matrices. 


THEOREM 1.4.3 


If R is the reduced row-echelon form of an n x n matrix A, then either R has a row of zeros or R is the identity matrix f. ; . 


Suppose that the reduced row-echelon form of A is 



^11 

^12 ‘ 

- ^1h 

^21 

^22 • 

'■ r 2n 

Oil 

r n2 ■ 

" r nn 


Either the last row in this matrix consists entirely of zeros or it does not. If not, the matrix contains no zero rows, and 
consequently each of the n rows has a leading entry of 1. Since these leading l's occur progressively farther to the right as we 
move down the matrix, each of these l's must occur on the main diagonal. Since the other entries in the same column as one of 
these l's are zero, R must be / . Thus, either R has a row of zeros or j? = / . 



























DEFINITION 


If A is a square matrix, and if a matrix B of the same size can be found such that AZ? = BA = L then A is said to be invertible 
and B is called an inverse of A. If no such matrix B can be found, then A is said to be singular. 


Verifying the Inverse Requirements 


The matrix 


since 


and 


5 = 


3 5 
1 2 


is an inverse of 



A5 = 


BA = 


2 

— 5~ 

'3 5' 


'1 O' 

_ - 1 

3_ 

_1 2_ 


_0 1_ 

'3 5' 

2 

-5' 


'1 O' 

_1 2_ 

_ - 1 

3_ 


_0 1_ 


♦ 


A Matrix with No Inverse 


The matrix 


is singular. To see why, let 


be any 3x3 matrix. The third column of BA is 


Thus 





'i 

A 

■ O' 




A 

= 

2 

5 

: 0 






3 

6 

; o 





*11 

*12 * 

'13 


B 

— 

*21 

*22 *23 




*31 

*32 *33 


*11 

b 

12 

*13 

'o' 


'o' 

*21 

*22 

*23 

0 

— 

0 

*31 

*32 

*33 

0 


0 





'1 

0 

o' 


BA*h 


0 

1 

0 






0 

0 

1 



♦ 


Properties of Inverses 

It is reasonable to ask whether an invertible matrix can have more than one inverse. The next theorem shows that the answer is 







































no— an invertible matrix has exactly one inverse. 


THEOREM 1.4.4 


If B and C are both inverses of the matrix A, then B—C- 


Proof Since B is an inverse of A, we have BA = /• Multiplying both sides on the right by C gives ( BA) C = 1C = C- But 
(BA) C = B(AC) = B1 = B, so C = B- 


As a consequence of this important result, we can now speak of “the” inverse of an invertible matrix. If A is invertible, then its 
inverse will be denoted by the symbol ^-1. Thus, 

AA~ l =I and A _1 A = / 

The inverse of A plays much the same role in matrix arithmetic that the reciprocal plays in the numerical relationships 
aa _i = 1 and a ~\- i = ]. 

In the next section we shall develop a method for finding inverses of invertible matrices of any size; however, the following 
theorem gives conditions under which a 2 x 2 matrix is invertible and provides a simple formula for the inverse. 


THEOREM 1.4.5 


The matrix 


A = 


a 

c 


b 

d 


is invertible if ad-bc^O, in which case the inverse is given by the formula 


A -1 


1 

ad — be 


d 
— c 



d 

ad — be 
e 

ad — be 


b 

ad — be 
a 

ad — be 


We leave it for the reader to verify that AA -1 = 1 2 ar| d A -1 A = Ij- 


THEOREM 1.4.6 


If A and B are invertible matrices of the same size, then AB is invertible and 

(A5) _1 =5 _1 A _1 
















If we can show that ( AB ) (B l ) — (B 1 ) (AB) — /, then we will have simultaneously shown that the matrix AB 

is invertible and that ( AB)~ 1 = 5 -1 A -1 - But {AB) = (5 -1 A -1 ) = A(55 -1 )A -1 = AL4 -1 = AA -1 = /. A similar argument 
shows that (B~ 1 A~ 1 )(AB) = I. 


Although we will not prove it, this result can be extended to include three or more factors; that is, 


A product of any number of invertible matrices is invertible, and the inverse of the product is the product of the inverses in 
the reverse order. 


Inverse of a Product 


Consider the matrices 


A = 


1 2 
1 3 


B = 


Applying the formula in Theorem 1.4.5, we obtain 


A~ l = 


Also, 


3 -2 

1 1 


5 _1 A _1 = 


B~ { - 


3 2 
2 2 


1 - 1 
> I 


AB = 


7 6 
9 8 


W _1 = 


4 

9 

2 


-3 

7 

2 


1-1' 

3 

-2' 


4 

-3" 

.- 1 l 

_ - 1 

1 _ 


9 

2 

7 

2 


Therefore, {AB ) = B^A - , as guaranteed by Theorem 1.4.6. 


Powers of a Matrix 

Next, we shall define powers of a square matrix and discuss their properties. 


DEFINITION 


If A is a square matrix, then we define the nonnegative integer powers of A to be 


A° = I A”=AA-A {n > 0) 

12 factors 

Moreover, if A is invertible, then we define the negative integer powers to be 


A= (A _1 ) M = A _1 A~ l -A _1 

n factoids 


Because this definition parallels that for real numbers, the usual laws of exponents hold. (We omit the details.) 





























THEOREM 1.4.7 


Laws of Exponents 

If A is a square matrix and r and s are integers, then 

A r A s = A r+s , (A r ) s = A n 

The next theorem provides some useful properties of negative exponents. 

THEOREM 1.4.8 

Laws of Exponents 

If A is an invertible matrix, then: 

(a) 

is invertible and (A~^) 1 = A 

(b) A n is invertible and (^4”) -1 = (j4 -1 ) ” for n — 0, 1, 2, .... 

( c ) For any nonzero scalar k, the matrix fcA is invertible and (kA ) -1 = 


Proof 


( a ) Since AA -1 = A _1 A = L the matrix a -1 is invertible and (j4 -1 ) 1 = A 


(b) This part is left as an exercise. 


(c) If k is any nonzero scalar, results (/) and ( m ) of Theorem 1.4.1 enable us to write 


&A) = (!)/ = / 


Similarly, (^4 Mob4) =1 so that kA is invertible and 0h4) i = jrA ^ 


Powers of a Matrix 






Let A and j 4 -1 be as in Example 7; that is, 


A = 


Then 


A 3 = 


A- 3 = (A- 1 ) = 


1 2 
1 3 


1 2 
1 3 


and 


A -1 = 


3 -2 
- 1 1 


" I " 1 2 " 

"1 

2' 


'll 

30' 

_[l 3_ 

_1 

3_ 


15 

41 _ 


3 -2 
1 1 


_ r 3 - 2 ' 

3-2' 


41 

-30' 

-1 1 

-1 1 


- 15 

11 


Polynomial Expressions Involving Matrices 

If A is a square matrix, say mxm, and if 


p{x) = flg + H-Uh/ 


( 1 ) 


is any polynomial, then we define 

p(A) =a$l A a\AA -h a„A” 

where I is the my,m identity matrix. In words, p (A) is the m x m matrix that results when A is substituted for x in 1 and ag is 
replaced by 


Matrix Polynomial 


If 


then 


p(x) = 2x 2 —3x+4 and 



p(A) = 2A 2 -3A f 4/ = 2 


-1 2 
0 3 


i2 


1 2 
0 3 


+ 4 


0 

1 


2 8 
0 18 


-3 6 
0 9 


-I- 


4 0 
0 4 


9 2 
0 13 


♦ 


Properties of the Transpose 

The next theorem lists the main properties of the transpose operation. 

THEOREM 1.4.9 

Properties of the Transpose 

If the sizes of the matrices are such that the stated operations can be performed, then 
(a) ((A) T ) T — A 








































(b) (A f B) T = A T + B T and (A-B) T = A T -B T 

( c ) (Aj4) 7 = kA 7 , where k is any scalar 

(d) (AB) T = B T A T 


If we keep in mind that transposing a matrix interchanges its rows and columns, parts (a), ( b ), and (c) should be self-evident. 
For example, part (a) states that interchanging rows and columns twice leaves a matrix unchanged; part ( b ) asserts that adding 
and then interchanging rows and columns yields the same result as first interchanging rows and columns and then adding; and 
part (c) asserts that multiplying by a scalar and then interchanging rows and columns yields the same result as first 
interchanging rows and columns and then multiplying by the scalar. Part ( d ) is not so obvious, so we give its proof. 


Proof (d) Let A = [fl;,■ ] and B= [bu] so that the products AB and B 7 A 7 can both be formed. We leave it for the 
reader to check that (AB) 7 and B 7 A 7 have the same size, namely n x m- Thus it only remains to show that corresponding 
entries of (AB) 7 and B 7 A 7 are the same; that is, 


((AB) 7 ) ii = (B 7 A 7 ) i 


( 2 ) 


Applying Formula 11 of Section 1.3 to the left side of this equation and using the definition of matrix 
multiplication, we obtain 


((AB) ) jj — (AB)p — ctj\b \j -I- ayib'ii 4- -I- ctpb r ^ 


(3) 


To evaluate the right side of 2, it will be convenient to let ab and bb denote the i yth entries of a 7 and 
B 7 , respectively, so 

a ij= a Ji ^ b ij = b A 

From these relationships and the definition of matrix multiplication, we obtain 

(B 7 A 7 )= b' a a[j -A b ! i2 a ! 2 j 4- - 4 b^a^ 


= b\iCtj\ 4 b 2l a j2 4.4 b n a }r 

= a 2 ib u 4 aj 2 b 2 i f - 4 


This, together with 3, proves 2. 


Although we shall not prove it, part ( d ) of this theorem can be extended to include three or more factors; that is, 


The transpose of a product of any number of matrices is equal to the product of their transposes in the reverse order. 


Remark Note the similarity between this result and the result following Theorem 1.4.6 about the inverse of a product of 
matrices. 



Invertibility of a Transpose 

The following theorem establishes a relationship between the inverse of an invertible matrix and the inverse of its transpose. 

THEOREM 1.4.10 

If A is an invertible matrix, then j[ T is also invertible and 

{A 7 )~ l = {A-') T ( 4 ) 


Proof We can prove the invertibility of A 7 and obtain 4 by showing that 

A 7 {A- l ) T = (A~ l ) T A 7 = I 

But from part ( d ) of Theorem 1.4.9 and the fact that i T = j, we have 

A r (A _1 ) T = (A _1 A) T = 1 7 =I 
(A ~ 1 ) 7 A 7 = (AA ~ 1 ) 7 = 1 7 = I 

which completes the proof. 


EXAMPLE 10 Verifying Theorem 1.4.10 


Consider the matrices 


Applying Theorem 1.4.5 yields 


A~ l = 


A = 


-5 -3 

2 1 


1 3 

-2 -5 


i 1 

C^- 1 ) = 




1 -2 
3 -5 


-5 2 
-3 1 


T ~ 1 

(A 7 ) = 


-2 
— 5 


As guaranteed by Theorem 1.4.10, these matrices satisfy 4. 


Exercise Set 1.4 


Click here for Just Ask! 




















2 

- 1 

3 


8 

-3 

-5 

A = 


0 

4 

5 

B = 

0 

1 

2 


~ 

-2 

1 

4 


4 

-7 

6 



'0 

-2 

3' 





C 

= 

1 

7 

4 

, a = A, 

& = 

-7 



3 

5 

9 






Show that 


(a) A I (B + C) = (A + B) + C 


(b) <AB)C = A(BC) 


(c) (a + b)C = aC I bC 


(d) a{B-C)=aB-aC 


Using the matrices and scalars in Exercise 1, verify that 

2 . 


(a) a{BC) = {aB)C = B{aC) 

(b) A(B - C) = AB - AC 

(c) (B f C)A = BA +■ CA 

(d) a{bC) = {ab)C 


Using the matrices and scalars in Exercise 1, verify that 

3. 

(a) (A T ) T = A 


(b) (A ±B) T = A T + B T 


(c) {aC) T = aC T 


(d) (AB) T = B T A T 


4. 


Use Theorem 1.4.5 to compute the inverses of the following matrices. 



(a) 


(b) 

(c) 

(d) 


A = 


B = 


C = 


D = 


3 1 
5 2 


2 -3 
4 4 


6 4 

-2 - 1 


2 0 
0 3 


Use the matrices A and B in Exercise 4 to verify that 

5 . 

(a) (j4 -1 ) _1 =A 


(b) (B 7 ) 1 = (5“ 1 ) 7 ’ 


Use the matrices A, B, and C in Exercise 4 to verify that 

6 . 

(a) (ABy 1 =B~ 1 A~ 1 

(b) {ABC)~ l =C~ l B- x A~ l 


In each part, use the given information to find A. 

7 . 


8 . 


Let A lpg^the n^atrijc 2 — 


2 0 
4 1 


Compute ^-3, aryl A 2 -2A I /• 

1 


(b) (1A ) -1 = 
Let A be the matrix 
'3 1 


( c ) " 
In each part, rind 


1 |" — 3 -1 

p(A)-5 2 


(d) 

r -1 2i 

(a) p(x) = x — 2- 

i 

in 


9 . 



(b) p(x) — 2x 2 — x I 1 

(c) p ( x ) = x 3 -2x-I 4 

Ld p\(x) = x 2 — 9> P2i x ) — * 4 3, and pi(x) — x - 3- 

(a) Show that p j (A) = p 2 (A)pi(A) for the matrix A in Exercise 9. 

(b) Show that p j (A) — P 2 (A)p 2 (A) for any square matrix A. 

Find the inverse of 

cos 9 sin# 

— sin# cos# 

Find the inverse of 

0 - O' 

a 22 - 0 

0 a nyi 

where an^22'“ a nn ^ 0- Show that A is invertible and find its inverse. 

Show that if a square matrix A satisfies A 2 — 3A f/ = 0, then A~^ = 31 — A 

14 . 


12. r 




Consider the matrix 


13 . 


A = 


a n 

0 

0 


15 . 

(a) Show that a matrix with a row of zeros cannot have an inverse. 

(b) Show that a matrix with a column of zeros cannot have an inverse. 


Is the sum of two invertible matrices necessarily invertible? 

16 . 

Let A and B be square matrices such that AB-0- Show that if A is invertible, then B-C- 


17 . 



18 . 


Let A, B, and 0 be 2 x 2 matrices. Assuming that A is invertible, find a matrix C such that 


is the inverse of the partitioned matrix 


L" 1 

0 

c 

A -i 

A 


ft 

A 


19 . 


(See Exercise 15 of the preceding section.) 

Use the result in Exercise 18 to find the inverses of the following matrices. 


(a) 


(b) 


1 1 

-1 1 

1 1 

1 1 


1 1 0 
0 1 0 
0 0 1 
0 0 0 


0 

0 

1 

-1 


0 

0 

1 

1 


0 

0 

1 

1 


20 . 

(a) Find a nonzero 3x3 matrix A such that A T = A- 

(b) Find a nonzero 3x3 matrix A such that a T = — A- 


A square matrix A is called symmetric if A T = A and skew-symmetric if A T = - A- Show that if B is a square matrix, then 

21 . 

(a) BE 7 and B I B T are symmetric 

(b) B — B T is skew-symmetric 


22 . 


23 . 


T 

If A is a square matrix and n is a positive integer, is it true that (A”) 


Let A be the matrix 

'1 0 f 
1 1 0 
0 1 1 

Determine whether A is invertible, and if so, find its inverse. 


'T' ri 

(A J ) ? Justify your answer. 






Hint Solve AX = l by equating corresponding entries on the two sides. 


Prove: 

24 . 


(a) part ( b ) of Theorem 1.4.1 


(b) part (0 of Theorem 1.4.1 


(c) part (m) of Theorem 1.4.1 


Apply parts (d) and (m) of Theorem 1.4.1 to the matrices A, B, and ( — 1) C to derive the result in part if). 


Prove Theorem 1.4.2. 

26 . 


Consider the laws of exponents A r A 5 = j 4 j " +5 and (A r ) 5 = A rs . 

27 . 


(a) Show that if A is any square matrix, then these laws are valid for all nonnegative integer values of r and s. 


(b) Show that if A is invertible, then these laws hold for all negative integer values of r and s. 


Show that if A is invertible and k is any nonzero scalar, then (kA) ” = k n A n for all integer values of n. 

28 . 


29 . 

(a) Show that if A is invertible and AB = AC, then B=C- 


(b) Explain why part (a) and Example 3 do not contradict one another. 


Prove part (c) of Theorem 1.4.1. 

30 . 

Hint Assume that A is m x n, B is n x P, and C is p x q. The i jth entry on the left side is 

Ijj = <3,1 [BC] i j < 3,2 [BC] 2 j 4-h <3,„ [BC] and the zyth entry on the right side is 

r ij = [4L5] j-jcij 4- [A6'] i 2 c 2 j 4 \-^B \ ipCpj- Verify that i 1 ,-^ = r,j- 


Discussion 

Discov&ry 


31 . 


Let A and B be square matrices with the same size. 




(a) Give an example in which (.44 B) 2 * A 2 4 2AB 4- B 2 . 

(b) Fill in the blank to create a matrix identity that is valid for all choices of A and B. 

(A 4 B) 2 = A 2 4 B 2 4 _. 


Let A and B be square matrices with the same size. 

32. 


(a) Give an example in which (A 4 B) (A — B) * A 2 — B 2 . 


(b) Let A and B be square matrices with the same size. Fill in the blank to create a matrix 
identity that is valid for all choices of A and B. (^4 f B)(A — B") = _• 


In the real number system the equation = \ has exactly two solutions. Find at least eight 
33* different 3x3 matrices that satisfy the equation = [y 

Hint Look for solutions in which all entries off the main diagonal are zero. 


A statement of the form “If p, then q” is logically equivalent to the statement “If not q, then not 
34. p.” (The second statement is called the logical contrapositive of the first.) For example, the 
logical contrapositive of the statement “If it is raining, then the ground is wet” is “If the ground 
is not wet, then it is not raining.” 


(a) Find the logical contrapositive of the following statement: If a T is singular, then A is 
singular. 

(b) Is the statement true or false? Explain. 


Let A and B be n x n matrices. Indicate whether the statement is always true or sometimes false. 
Justify each answer. 


(a) (AB) 2 — A 2 B 2 

(b) (A-B) 2 = (B-A) 2 

(c) {AB~ l ){BA~ l ) =1„ 


35. 




(d) AB* BA- 


Assuming that all matrices are n x n and invertible, solve for D. 

abc t dba t c=ab t 


Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 



1-5 

ELEMENTARY MATRICES 
AND A METHOD FOR 
FINDING a~ l 


In this section we shall develop an algorithm for finding the inverse of an 
invertible matrix. We shall also discuss some of the basic properties of 
invertible matrices. 


We begin with the definition of a special type of matrix that can be used to carry out an elementary row operation by matrix 
multiplication. 


DEFINITION 


An hxb matrix is called an elementary matrix if it can be obtained from the n x n identity matrix / by performing a single 
elementary row operation. 


Elementary Matrices and Row Operations 


Listed below are four elementary matrices and the operations that produce them. 


1 0 
0 -3 


10 0 0 
0 0 0 1 
0 0 10 
0 10 0 


'1 0 3' 


1 

O 

O 

0 1 0 


0 1 0 

1 - 

O 

O 


0 0 1 


T 

T 

T 

Multiply the 

Interchange the 

Add 3 times 

second row of 

second and fourth 

the third row of 

^2 by — 3 . 

rows of /4 . 

12 to the first row. 


T 

Multiply the 
Erst row of 
ll by 1 . 


♦ 


When a matrix A is multiplied on the left by an elementary matrix E, the effect is to perform an elementary row operation on A. 
This is the content of the following theorem, the proof of which is left for the exercises. 

THEOREM 1.5.1 


Row Operations by Matrix Multiplication 

If the elementary matrix E results from performing a certain row operation on [ and if A is an my..n matrix, then the product 
EA is the matrix that results when this same row operation is performed on A. 




















Using Elementary Matrices 


Consider the matrix 


and consider the elementary matrix 




0 

1 

0 


which results from adding 3 times the first row of to the third row. 


EA = 


1 

2 

4 



2 3 

3 6 

4 0 

O' 

0 

1 

The product EA is 

2 3' 

3 6 
10 9 


which is precisely the same matrix that results when we add 3 times the first row of A to the third row. 


♦ 


Remark 

Theorem 1.5.1 is primarily of theoretical interest and will be used for developing some results about matrices and systems of 
linear equations. Computationally, it is preferable to perform row operations directly rather than multiplying on the left by an 
elementary matrix. 


If an elementary row operation is applied to an identity matrix 7 to produce an elementary matrix E, then there is a second row 
operation that, when applied to E, produces 7 back again. For example, if E is obtained by multiplying the zth row of 7 by a 
nonzero constant c, then 7 can be recovered if the zth row of E is multiplied by ] j C - The various possibilities are listed in Table 1. 
The operations on the right side of this table are called the inverse operations of the corresponding operations on the left. 


Table 1 


Row Operation on 7 That Produces E 

Row Operation on E That Reproduces I 

Multiply row i by c ^ 0 

Multiply row i by 1 / c 

Interchange rows i and j 

Interchange rows i and j 

Add c times row i to row j 

Add — c times row i to row j 


Row Operations and Inverse Row Operations 


In each of the following, an elementary row operation is applied to the 2 x 2 identity matrix to obtain an elementary matrix E, 
then E is restored to the identity matrix by applying the inverse row operation. 


















'1 o ' 


'1 o ' 


'1 o ' 

.0 


_0 7_ 


_0 1 _ 


T 


T 


Multiply the Multiply the 

second row second row 

by 7. 

by^. 


'1 O ' 


'0 1 " 


'1 o ' 

_0 1 _ 


_1 0 _ 


_0 1 _ 


T T 

Interchange the Interchange the 


first and second first and second 
rows. rows. 


-1 

o 


1 

L-h 
_1 


'1 o' 

1- 

o 

1_ 


1 

o 
_ 1 


1 

o 

1 _ 


T T 


Add 5 times 
the second row 
to the Erst. 


Add — 5 times 
the second row 
to the Erst. 


* 


The next theorem gives an important property of elementary matrices. 


THEOREM 1.5.2 


Every elementary matrix is invertible, and the inverse is also an elementary matrix. 


Proof If E is an elementary matrix, then E results from performing some row operation on I. Let be the matrix that results 
when the inverse of this operation is performed on I. Applying Theorem 1.5.1 and using the fact that inverse row operations 
cancel the effect of each other, it follows that 


EqE — l and EEq — l 

Thus, the elementary matrix £ 0 is the inverse of E. 

■ 

The next theorem establishes some fundamental relationships among invertibility, homogeneous linear systems, reduced 
row-echelon forms, and elementary matrices. These results are extremely important and will be used many times in later sections. 


THEOREM 1.5.3 


Equivalent Statements 

If A is an n matrix, then the following statements are equivalent, that is, all true or all false. 
(a) A is invertible. 






















(b) Ax = 0 has only the trivial solution. 


(c) The reduced row-echelon form of A is / H . 


(d) A is expressible as a product of elementary matrices. 


Proof We shall prove the equivalence by establishing the chain of implications: (a) ( b ) ( c ) (d) (a). 

Assume A is invertible and let xq be any solution of Ax = 0; thus Axq = 0- Multiplying both sides of this equation by 
the matrix a~ 1 gives (j4x 0 ) =A~ 1 (h or (A^A'jxq = 0, or ; X|] = 0, or XQ = 0- Thus, Ax = 0 has only the trivial 
solution. 


=> (tr) Let = 0 be the matrix form of the system 


auxi + a 12 x 2 H-1- «1h*h = 0 

a 2l x 1 T <322*2 H- V a 2n x n = 0 

a H l*l + a H 2*2 H-f = 0 


and assume that the system has only the trivial solution. If we solve by Gauss-Jordan 
elimination, then the system of equations corresponding to the reduced row-echelon form of 
the augmented matrix will be 


Thus the augmented matrix 


*1 


-0 

*2 

-0 



7 l ' v 3 = 0 

All 

«12 

"" a 1 h 

<*21 

« 22 

""" a 2 H 

fl H l 

a m2 

a nn 


for 1 can be reduced to the augmented matrix 

" 1 0 0 — 0 0 
0 1 0 - 0 0 
0 0 1-00 


( 2 ) 


0 0 0 - 1 0 


for 2 by a sequence of elementary row operations. If we disregard the last column (of zeros) in each of these matrices, 
we can conclude that the reduced row-echelon form of A is / . 

1 H 


Assume that the reduced row-echelon form of A is / , so that A can be reduced to / by a finite sequence of elementary 
row operations. By Theorem 1.5.1, each of these operations can be accomplished by multiplying on the left by an 
appropriate elementary matrix. Thus we can find elementary matrices • • •, such that 







S^“EiS[A — I n 


( 3 ) 


By Theorem 1.5.2, E\, S 2 / —> E are invertible. Multiplying both sides of Equation 3 on the left 
successively by g~ { , ..., g~ { , g~ l we obtain 

K- A 1 

A = BpBp~BpI„ = BpI% l -Bp (4) 

By Theorem 1.5.2, this equation expresses A as a product of elementary matrices. 

= (a) If A is a product of elementary matrices, then from Theorems Theorem 1.4.6 and Theorem 1.5.2, the matrix A is a 
product of invertible matrices and hence is invertible. 


Row Equivalence 

If a matrix B can be obtained from a matrix A by performing a finite sequence of elementary row operations, then obviously we 
can get from B back to A by performing the inverses of these elementary row operations in reverse order. Matrices that can be 
obtained from one another by a finite sequence of elementary row operations are said to be row equivalent. With this 
terminology, it follows from parts (a) and (c) of Theorem 3 that an n x n matrix A is invertible if and only if it is row equivalent to 
the n x n identity matrix. 

A Method for Inverting Matrices 

As our first application of Theorem 3, we shall establish a method for determining the inverse of an invertible matrix. 

Multiplying 3 on the right by a -1 yields 


A — E%-‘E2E\I yi (5) 

which tells us that a -1 can be obtained by multiplying / successively on the left by the elementary matrices g^, g^, ..., g^. 
Since each multiplication on the left by one of these elementary matrices performs a row operation, it follows, by comparing 
Equations 3 and 5, that the sequence of row operations that reduces A to / will reduce l n to 1. Thus we have the following 
result: 


To find the inverse of an invertible matrix A, we must find a sequence of elementary row operations that reduces A to the 
identity and then perform this same sequence of operations on / to obtain A -1 - 


A simple method for carrying out this procedure is given in the following example. 


EXAMP Using Row Operations to Find A -1 


Find the inverse of 


A = 


3 

3 

8 


Solution 






We want to reduce A to the identity matrix by row operations and simultaneously apply these operations to I to produce A 1 • To 
accomplish this we shall adjoin the identity matrix to the right side of A, thereby producing a matrix of the form 

fill/1 


Then we shall apply row operations to this matrix until the left side is reduced to 7; these operations will convert the right side to 
so the final matrix will have the form 

[7U4- 1 ] 

The computations are as follows: 


1 2 3 

2 5 3 

I 0 8 

I 2 3 

0 1 -3 

0 -2 5 

I 2 3 

0 \ -3 

0 0 -1 

I 2 3 

« [ —3 

0 0 1 

12 0 
0 l 0 

0 0 1 

10 0 
0 t 0 

0 0 1 


l 0 0 

0 [ 0 
0 0 1 

l 0 0 

-2 1 0 
- 1 0 1 

I 0 0 

-2 1 0 
-5 2 1 

1 0 0 
-2 I 0 
5 -2 -1 

-14 6 3 

13 —5 -3 

5 -2 -1 

-40 16 9 

13 -5 —3 
5 -2 -1 


Wc ndd'cit —2 Times die 

KTihc fict+tnJ jiid I times 
llic IifM mvv (a the lliml 


Wu adtkd 2 limes the 
tvw Eu eItj iJiird. 


W L j ri iLEll i|iIi ifii I h e 
I li i rd nm b\ I. 


We iijiteil J lirrsi’s ihe lliml 
rpw so she Kevunil :nid min;-; 
the third low to the Jim, 


We Htldetl — 2 Times the 
M'ccini) mw Eii tlie I i r--i 


Thus, 


A -1 = 

' -40 

16 

9' 

13 

— 5 

-3 


5 

-2 

-1 


♦ 


Often it will not be known in advance whether a given matrix is invertible. If an n x n matrix A is not invertible, then it cannot be 
reduced to / by elementary row operations [part (c) of Theorem 3]. Stated another way, the reduced row-echelon form of A has 
at least one row of zeros. Thus, if the procedure in the last example is attempted on a matrix that is not invertible, then at some 
point in the computations a row of zeros will occur on the left side. It can then be concluded that the given matrix is not 
invertible, and the computations can be stopped. 


Showing That a Matrix Is Not Invertible 


Consider the matrix 


























Applying the procedure of Example 4 yields 


A = 


1 6 4 

2 4-1 

-12 5 


1 

6 

4 

i 

0 

0 " 

2 

4 

-1 

0 

1 

0 

-1 

*> 

5 

0 

(1 

1 

1 

fj 

4 

1 

0 

o’ 

0 

-K 

-y 

_ 2 

1 

0 

0 

K 

9 

l 

f) 

1 

1 

6 

4 

I 

{) 

o" 

0 

-.8 

-y 

_2 

1 

0 

0 

0 

a 

-I 

! 

1 


Wc ackted simcfi tht first 
tow tn ! fit 1 sccfflul unit mideil 
llk r flfrl fiJW [Oi [Pit! llliftl, 


Wc 4ictdc<f TllC 

mtclufuI rt'AH- Ui 


lh-j iliirtL 


Since we have obtained a row of zeros on the left side, A is not invertible. 


♦ 


EXAMPLE 6 A Consequence of Invertibility 


In Example 4 we showed that 


A = 


1 2 3 

2 5 3 
1 0 8 


is an invertible matrix. From Theorem 3, it follows that the homogeneous system 

xi I- 2x2 + 3*3 = 0 
2xi + 5*2 4- 3x3 — 0 
xi + 8 x 3 = 0 

has only the trivial solution. 


♦ 


Exercise Set 1.5 



Click here for Just Ask! 


Which of the following are elementary matrices? 

1. 


(a) 


1 0 

-5 1 


-5 1 

1 0 


(b) 























(C) 


1 

0 



(d) 


0 0 
0 1 
1 0 


1 

0 

0 


(e) 


1 1 0 
0 0 1 
0 0 0 


(f) 


1 0 0 
0 1 9 
0 0 1 


(g) 


2 0 0 2 
0 10 0 
0 0 10 
0 0 0 1 


Find a row operation that will restore the given elementary matrix to an identity matrix. 

2 . 


(a) 


1 0 
-3 1 


(b) 


1 0 0 
0 1 0 
0 0 3 


(c) 


0 0 0 1 
0 10 0 
0 0 10 
10 0 0 


(d) 


0 


0 1 
0 0 
0 0 


0 0 
1 0 
0 1 


3. 


Consider the matrices 






3 4 1 


8 1 5 


3 4 1 

2 -7 -1 

B = 

2 -7 -1 

C = 

2-7-1 

8 1 5 


3 4 1 


2-7 3 


Find elementary matrices E\> E 2 -> Ey anc * S 4 such that 

(a) E\A = B 

(b) E 2 B = A 

(c) E 3 A = C 

(d) E 4 C = A 


In Exercise 3 is it possible to find an elementary matrix E such that EB — C? Justify your answer. 

4. 


If a 2 x 2 matrix is multiplied on the left by the given matrices, what elementary row operation is performed on that matrix? 

5. 


(a) 


0 1 
1 0 


(b) 


2 

0 


0 

3 


(c) 


1 0 

-2 1 


In Exercises 6-8 use the method shown in Examples Example 4 and Example 5 to find the inverse of the given matrix if the 
matrix is invertible, and check your answer by multiplication. 


6. 


(a) 


1 4 

2 7 


(b) 


-3 6 
4 5 


6 -4 
-3 2 


(c) 


7, 


(a) 


3 4-1 

1 0 3 

2 5-4 


(b) 


- 1 

3 

-4 

2 

4 

1 

-4 

2 

-9 


(c) 


1 0 1 
0 1 1 
1 1 0 


(d) 


2 6 6 
2 7 6 
2 7 7 


(e) 


1 0 1 

-1 1 1 

0 1 0 


8 . 


(a) 


I _ 1 

5 5 

1 J_ 

5 10 

4 J_ 

5 10 


(b) 


3/2 0 

- 4/2 0 

0 0 1 


(c) 


1 

0 

0 

0 

1 

3 

0 

0 

1 

3 

5 

0 

1 

3 

5 

7 


(d) 


-8 17 2 j 

4 0 | -9 

0 0 0 0 

-1 13 4 2 




(e) 


0 0 2 0 

1 0 0 1 

0-13 0 

2 15-3 


Find the inverse of each of the following 4x4 matrices, where ky k$ and k are all nonzero. 


(a) 


ki 0 0 0 

0 k 2 0 0 

0 0 k 3 0 

0 0 0 £4 


(b) 


0 0 0 *1 

0 0 k 2 0 
0 0 0 
£4000 


(C) 


k 0 0 0 
1 £ 0 0 
0 1 £ 0 
0 0 1 jt 


Consider the matrix 

10 . 



0 

2 


(a) Find elementary matrices E\ and E 3 suc h that E 3 E\A = /• 

(b) Write j 4 _1 as a product of two elementary matrices. 


(c) Write A as a product of two elementary matrices. 


In each part, perform the stated row operation on 

'2-1 O' 

4 5-3 

1 -4 7 

by multiplying A on the left by a suitable elementary matrix. Check your answer in each case by performing the row 
operation directly on A. 











(a) Interchange the first and third rows. 


(b) Multiply the second row by -i. 


(c) Add twice the second row to the first row. 


Write the matrix 


12 . 


3 -2 
3 - 1 


as a product of elementary matrices. 

Note There is more than one correct solution. 


Let 


13. 


1 0 
0 4 
0 0 


(a) Find elementary matrices E\, E^, and such that B 2 B 2 B 1 A = 1 3 - 


(b) Write A as a product of elementary matrices. 


14. 


Express the matrix 


A = 


0 17 8 

1 3 3 8 

2 -5 1 -8 


in the form A = E F G & where E, F, and G are elementary matrices and R is in row-echelon form. 


Show that if 


15. 


A = 


1 0 0 
0 1 0 
a h c 


is an elementary matrix, then at least one entry in the third row must be a zero. 


Show that 


16. 


A = 


0 a 0 0 0 

b 0 c 0 0 

0 d 0 e 0 

0 0 / 0 g 

0 0 0 h 0 


is not invertible for any values of the entries. 


17. 


Prove that if A is an m x n matrix, there is an invertible matrix C such that CA is in reduced row-echelon form. 










Prove that if A is an invertible matrix and B is row equivalent to A, then B is also invertible. 


18. 


19. 

(a) Prove: If A and B are m x n matrices, then A and B are row equivalent if and only if A and B have the same reduced 
row-echelon form. 


(b) Show that A and B are row equivalent, and find a sequence of elementary row operations that produces B from A. 



'1 2 3' 


1 

O 

U-i 

A = 

1 4 1 

B = 

0 2-2 


2 1 9 


1 1 4 


Prove Theorem 1.5.1. 

20 . 


Discussion 

Discov&ry 


Suppose that A is some unknown invertible matrix, but you know of a sequence of elementary row 
21. operations that produces the identity matrix when applied in succession to A. Explain how you can 
use the known information to find A. 


Indicate whether the statement is always true or sometimes false. Justify your answer with a 
22. logical argument or a counterexample. 


(a) Every square matrix can be expressed as a product of elementary matrices. 


(b) The product of two elementary matrices is an elementary matrix. 


(c) If A is invertible and a multiple of the first row of A is added to the second row, then the 
resulting matrix is invertible. 


(d) If A is invertible and AB = 0, then it must be true that B = 0- 


Indicate whether the statement is always true or sometimes false. Justify your answer with a 
logical argument or a counterexample. 


(a) If A is a singular nxn matrix, then Ax = 0 has infinitely many solutions. 


(b) If A is a singular nxn matrix, then the reduced row-echelon form of A has at least one row 
of zeros. 








(c) If j 4 -1 is expressible as a product of elementary matrices, then the homogeneous linear 
system Ax = 0 has only the trivial solution. 

(d) If A is a singular n x n matrix, and B results by interchanging two rows of A, then B may or 
may not be singular. 


24. 


Do you think that there is a 2 x 2 matrix A such that 


a b 


~b 

d~ 

i 

_i 


a 

c 


for all values of a, b, c, and d ? Explain your reasoning. 
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1.6 

FURTHER RESULTS ON 
SYSTEMS OF 
EQUATIONS AND 
INVERTIBILITY 


In this section we shall establish more results about systems of linear equations 
and invertibility of matrices. Our work will lead to a new method for solving n 
equations in n unknowns. 


A Basic Theorem 

In Section 1.1 we made the statement (based on Figure 1.1.1) that every linear system has no solutions, or has one solution, or has 
infinitely many solutions. We are now in a position to prove this fundamental result. 


THEOREM 1.6.1 


Every system of linear equations has no solutions, or has exactly one solution, or has infinitely many solutions. 


Proof If Ax = lj is a system of linear equations, exactly one of the following is true: (a) the system has no solutions, (b) the system 
has exactly one solution, or (c) the system has more than one solution. The proof will be complete if we can show that the system 
has infinitely many solutions in case (c). 


Assume that j4x = b has more than one solution, and let xo = xj — X 2 , where xj and X 2 are any two distinct solutions. Because xi 
and X? are distinct, the matrix xo is nonzero; moreover, 

j4xq = j4(xi — X 2 ) = j4xi — A\2 = b — b = 0 


If we now let k be any scalar, then 


j4(xH £xo) = j4xj + j4(£xo) = j4xi 4- £(j4xo) 
= b H £0 = b 4 0 = b 


But this says that X j -\ £iq is a solution of j4x = b- Since xo is nonzero and there are infinitely many choices for k, the system 
j4x = b has infinitely many solutions. 


Solving Linear Systems by Matrix Inversion 

Thus far, we have studied two methods for solving linear systems: Gaussian elimination and Gauss-Jordan elimination. The 
following theorem provides a new method for solving certain linear systems. 


THEOREM 1.6.2 


If A is an invertible nxn matrix, then for each « x 1 matrix b, the system of equations j4x = b has exactly one solution, namely, 

x = A~ 1 b- 





Since A(A — b, it follows that x = A -1 b is a solution of Ax = fo. To show that this is the only solution, we will assume 
that xg is an arbitrary solution and then show that xo must be the solution A - 1 b- 

If iq is any solution, then Axg = b- Multiplying both sides by A _1 > we obtain = A - 1 b- 


Solution of a Linear System Using /p 1 


Consider the system of linear equations 

xj 4 2x2 + 3x3 = 5 
2xj 4 5x2 + 3x3 = 3 
xi 4 8 x 3 — 17 

In matrix form this system can be written as Ax = b» where 

5 
3 

17 



l 

1—«■ 

[-0 

OO 

_l 


■*f 


A = 

2 5 3 

, X = 

*2 

b = 


l 

CO 

O 

i—1 

_1 


*3 



In Example 4 of the preceding section, we showed that A is invertible and 


A -1 


40 16 9 

13 -5 -3 
5-2-1 


By Theorem 1.6.2, the solution of the system is 


' -40 

16 

9' 

' 5' 


r 

13 

5 

1 1 

-3 

- 1 

3 

17 

— 

- 1 

2 


or xi = T X 2 = — 1 > X 3 = 2 - 


♦ 


Remark 

Note that the method of Example 1 applies only when the system has as many equations as unknowns and the coefficient matrix is 
invertible. This method is less efficient, computationally, than Gaussian elimination, but it is important in the analysis of equations 
involving matrices. 

Linear Systems with a Common Coefficient Matrix 

Frequently, one is concerned with solving a sequence of systems 

Ac = bi, Ax = ti2> AK = b 3 ,.*^ A\ = bfr 
each of which has the same square coefficient matrix A. If A is invertible, then the solutions 

x\=A~ 1 b\, X? =A~ l h 2 ? X 2 =A~ 1 b 3 , xfc = A _1 bfc 

can be obtained with one matrix inversion and k matrix multiplications. Once again, however, a more efficient method is to form the 
matrix 


[ 2 l|bi |b 2 l—Ibfc] ^ 

in which the coefficient matrix A is “augmented” by all k of the matrices bj, b 2 »__ bjt» and then reduce 1 to reduced row-echelon 

form by Gauss-Jordan elimination. In this way we can solve all k systems at once. This method has the added advantage that it 
applies even when A is not invertible. 


















Solving Two Linear Systems at Once 


Solve the systems 

(a) 

*1 4 - 2 x 2 + 3*3 


2xi 

4- 5*2 4- 3*3 


*1 

4 - 8 x 3 

(b) 

*1 

4 2x2 4- 3*3 


2xi 

4 5x2 4- 3*3 


*1 

4 - 8 x 3 


Solution 

The two systems have the same coefficient matrix. If we augment this coefficient matrix with the columns of constants on the right 
sides of these systems, we obtain 


" 1 

2 

3 

4 

ll 

2 

5 

3 

5 

6 

1 

0 

3 

9 

-6 


Reducing this matrix to reduced row-echelon form yields (verify) 



(} 

0 

1 

2~ 

0 

1 

0 

(} 

1 

0 

0 

1 

1 

-1 


It follows from the last two columns that the solution of system (a) is = 1, X 2 = 0’ X 3 = 1 an d the solution of system (b) is = 2> 

*2 = I* *3 = - 1- 

♦ 

Properties of Invertible Matrices 

Up to now, to show that an M x n matrix A is invertible, it has been necessary to find an M x n matrix B such that 

AB = I and BA = I 

The next theorem shows that if we produce an w x n matrix B satisfying either condition, then the other condition holds 
automatically. 


THEOREM 1.6.3 


Let Abe a square matrix. 

(a) If B is a square matrix satisfying BA = L then B = A ~ 1 - 


(b) If B is a square matrix satisfying AB = L then B = A ^ 













We shall prove part (a) and leave part (b) as an exercise. 


)f (a) Assume that BA = 1- If we can show that A is invertible, the proof can be completed by multiplying BA = I on both sides 
by j 4 _1 to obtain 


BAA~ l =lA~ l or BI = IA~ l or B = A~ X 

To show that A is invertible, it suffices to show that the system ^=0 has only the trivial solution (see 
Theorem 3). Let xq be any solution of this system. If we multiply both sides of ^ 0=0 on the left by B, we 
obtain bAxq = bo or /x 0 =0 or Xo =0- Thus, the system of equations = 0 has only the trivial solution. 


We are now in a position to add two more statements that are equivalent to the four given in Theorem 3. 


THEOREM 1.6.4 

Equivalent Statements 

If A is an n x n matrix, then the following are equivalent. 

(a) A is invertible. 

(b) Ax = 0 has only the trivial solution. 

(c) The reduced row-echelon form of A is / . 

(d) A is expressible as a product of elementary matrices. 

(e) Ax = ti is consistent for every n x 1 matrix b. 

(f) Ax = ti h as exactly one solution for every n x 1 matrix b. 


Since we proved in Theorem 3 that ( a ), ( b ), (c), and (. d ) are equivalent, it will be sufficient to prove that (a) (f) ( e ) 

(a). 

(a) = This was already proved in Theorem 1.6.2. 

3 ( e ) This is self-evident: If Ax = ti has exactly one solution for every w x 1 matrix b , then Ax. = b is consistent for every w x 1 
matrix b. 

(i e) => (a) If the system Ax — h is consistent for every M x 1 matrix b , then in particular, the systems 




1 


0 


0 


0 


1 


0 

Ax = 

0 

, Ax = 

0 

Ax = 

0 


0 


0 


1 


are consistent. Let x l7 x 2 , x H be solutions of the respective systems, and let us form an 
matrix C having these solutions as columns. Thus C has the form 

C = [xilx2l-lx„] 


As discussed in Section 1.3, the successive columns of the product AC will be 

Axi, Ax 2 , Ax„ 

Thus 


AC = [Axi lAjql—IA x h ] = 


1 0 
0 1 


0 0-0 


0 0-1 

By part ( b ) of Theorem 1.6.3, it follows that C — A -1 - Thus, A is invertible. 


= 1 


We know from earlier work that invertible matrix factors produce an invertible product. The following theorem, which will be 
proved later, looks at the converse: It shows that if the product of square matrices is invertible, then the factors themselves must be 
invertible. 


THEOREM 1.6.5 


Let A and B be square matrices of the same size. If AB is invertible, then A and B must also be invertible. 


In our later work the following fundamental problem will occur frequently in various contexts. 


A Fundamental Problem: Let A be a fixed mxn matrix. Find all m * 1 matrices b such that the system of equations Ax = b is 
consistent. 


If A is an invertible matrix, Theorem 1.6.2 completely solves this problem by asserting that for every mx.\ matrix b, the linear 
system Ax = b has the unique solution x = A _1 b- If A is not square, or if A is square but not invertible, then Theorem 1.6.2 does not 
apply. In these cases the matrix b must usually satisfy certain conditions in order for Ax = b to be consistent. The following example 
illustrates how the elimination methods of Section 1.2 can be used to determine such conditions. 


Determining Consistency by Elimination 


What conditions must and hi satisfy in order for the system of equations 

xi fx 2 +- 2x3 =&i 
xi A *3=^2 
2xi 3x3 =^3 


to be consistent? 













Solution 


The augmented matrix is 

'l 1 2 Ai 

1 0 1 b 2 

2 1 3 63 

which can be reduced to row-echelon form as follows: 


1 1 2 *i 

0 -1 -1 6 2 -6 i 

0 -1 -1 * 3 — 2*i 

1 1 2 bi 

0 1 1 6i-6 2 

0 -1 -1 63 — 261 

112 61 

0 1 1 6 i-6 2 

0 0 0 63 — 62 — 61 


— 1 times the Erst row was 
added to the second and — 2 
times the Erst row was added 
to the third. 

The second row was 
multiplied by — 1. 

The second row was 
added to the third. 


It is now evident from the third row in the matrix that the system has a solution if and only if b 2 , and 63 satisfy the condition 

63 — 62 — 61 = 0 or 63 = 61 H 62 

To express this condition another way, ^4* = I, is consistent if and only if 6 is a matrix of the form 


b = 


*1 

61 -I- 62 


where b\ and h 2 are arbitrary. 


EXAMPLE 4 Determining Consistency by Elimination 


What conditions must b\> b 2 , and 63 satisfy in order for the system of equations 

xi -I- 2x2 + 3x3 =&i 
2xi d 5x2 + 3x3 = &2 
xi + 8 x 3 = 63 

to be consistent? 

Solution 

The augmented matrix is 

'l 2 3 61" 

2 5 3 62 

1 0 8 63 

Reducing this to reduced row-echelon form yields (verify) 
















In this case there are no restrictions on an <3 *3; that is, the given system Ax = b has the unique solution 

? L 'l = — 40*i -h 16i 2 -1“ ^* 3 , = 13*1 — 5*2 — 3*3, ^3 = 5*1 — 2*2 — *3 


for all b. 


Remark Because the system Ax = b in the preceding example is consistent for all b , it follows from Theorem 1 . 6.4 that A is 
invertible. We leave it for the reader to verify that the formulas in ( 3 ) can also be obtained by calculating x — A _1 b- 


Exercise Set 1.6 



Click here for Just Ask! 


In Exercises 1-8 solve the system by inverting the coefficient matrix and using Theorem 1 . 6 . 2 . 

xi f x 2 = 2 
5xi f 6 x 2 = 9 


4 xi - 3*2 = - 3 
2xi — 5 x 2 = 9 

x i + 3x2 + X3 = 4 

2xi + 2x2 + X3 = — 1 

2xi + 3x2 + X3 = 3 

5xi + 3x2 + 2x3 =4 
3x i 4- 3x2 + 2x3 = 2 
X 2 + x 3 = 5 

x f y+ z= 5 
5 * x+7-4z=10 

— 4x f y-V z= 0 


— x — 2 v — 3 z = 0 
6 * w \- x T Ay -(- 4 z =10 
w f 3 x + 7 yH 9 z = 4 
— w — 2x — 4 y — 6 z — 6 

3 xi + 5 x 2 =b\ 

7 ‘ xi f 2x2 = ^2 

xi + 2x2 4- 3x3 = &i 

g 

2xi + 5 x 2 + 5 x 3 = 

3xi b 5x2 b Sx3 = bj 

Solve the following general system by inverting the coefficient matrix and using Theorem 1.6.2. 

xi -I- 2x2 + 3x3 = &i 
xi- X2 +X3 = ^2 
xi 4- X2 =&3 


9 . 



Use the resulting formulas to find the solution if 


(a) b\ = - 1 , * 2 = 3, *3 =4 

(b) =5, i 2 = 0, i 3 = 0 

( c ) = — 1> b'i = — 1’ ^3 = 3 


Solve the three systems in Exercise 9 using the method of Example 2. 

10 . 

In Exercises 11-14 use the method of Example 2 to solve the systems in all parts simultaneously. 

xi - 5x2 = ii 
3xi 4 2 x 2 =H 


(a) &i = h &2 = 4 

(b) bi= - Zb 2 = 5 


— xi I 4x2 + *3=^1 
xi 4 9 x 2 - 2x 3 =h 
6*1 I 4*2 — S*3 = £3 


(a) = 0, .£'2 = 1> b 3 = 0 


(b) b 3 = -3,h 2 =4,h 3 = -5 



4*i -7*2 =b\ 
13 ‘ * i 4 2*2 = bj 


(a) = 0> bj = 1 

(b) ij = _ 4 , b2 = 6 

(c) ij = _ bb 2 = 3 

(d) ij = - 5 ,b 2 = 1 

*1 4 3*2 4 5*3 = &i 
14 ‘ -* i - 2*2 =£'2 

2 *i 4 5*2 4 4*3 = &3 

(a) i 1 = l,i 2 = 0,i 3 = -1 

(b) i 1= 0,6 2 = l,i 3 = l 

(c) b\ = - hb 2 = - 1,63 = 0 

The method of Example 2 can be used for linear systems with infinitely many solutions. Use that method to solve the systems 
15 . in both parts at the same time. 


( a ) *1 — 2*2 f *3 = — 2 

2*i — 5*2 I- *3 = 1 

3*i — 7*2 I 2*3 = — 1 

(b) *1 — 2*2 4 *3 = 1 

2*i — 5*2 f *3 = — 1 

3*i — 7*2 I 2*3 — 0 

In Exercises 16-19 find conditions that the b's must satisfy for the system to be consistent. 
6*1 - 4*2 =b\ 

16 ‘ 3*i - 2*2 =^2 

*1 — 2*2 -I- 5*3 = b\ 

4*i — 5*2 -I- 8*3 = &2 
— 3*1 4 3*2 — 3*3 = &3 




*1 - 2x2 — *3 =b\ 

— 4*i 4 5x2 +- 2x2 = 

— 4*i 4- 7*2 -I- 4*3 = &2 


*i — X2 +■ 3x2 f 2xa = &i 

19. 

— 2xi+ *2 I 5*3 f *4 = &2 

— 3*i + 2*2 I 2*3— *4 = &3 
4*i— 3*2 I *3 f 3*4 = ^4 

Consider the matrices 


'2 

1 

2' 



■*r 

2 

2 

-2 

and 

x = 

*2 

3 

1 

1 



*3 


(a) Show that the equation Ax = x can be rewritten as (A — l)x = 0 and use this result to solve Ax = x for x. 

(b) Solve j4x = 4x- 


Solve the following matrix equation for X. 


'1 

- 1 

r 


'2 

- 1 

5 

7 

8 ' 

2 

3 

0 

X = 

4 

0 

-3 

0 

1 

0 

2 

- 1 


3 

5 

-7 

2 

1 


In each part, determine whether the homogeneous system has a nontrivial solution (without using pencil and paper); then state 

22. whether the given matrix is invertible. 


(a) 2* i 4 *2 — 3*3 4 - *4 = 0 

5*2 4 4*3 4 3*4 = 0 
*3 4 2*4 = 0 
3*4 = 0 

(b) 5*i 4 *2 4 4*3 4 *4 = 0 

2*3 — *4 = 0 
*3 4 *4 = 0 
7*4 = 0 


2 1-3 

0 5 4 

0 0 1 

0 0 0 


5 14 1 
0 0 2 -1 
0 0 1 1 
0 0 0 7 


1 

3 

2 

3 


Let j4x = 0 be a homogeneous system of n linear equations in n unknowns that has only the trivial solution. Show that if k is 

23. any positive integer, then the system _ q also has only the trivial solution. 


Let j4x = 0 be a homogeneous system of n linear equations in n unknowns, and let Q be an invertible n x n matrix. Show that 

24. j4x = 0 has just the trivial solution if and only if (QA) x = 0 has just the trivial solution. 

Let j4x = 1) be any consistent system of linear equations, and let *i be a fixed solution. Show that every solution to the system 

25. can be written in the form x = xj f xq. where xq is a solution 4 X = 0- Show also that every matrix of this form is a solution. 





Use part (a) of Theorem 1.6.3 to prove part (b). 


26 . 

What restrictions must be placed on x and y for the following matrices to be invertible? 

27. 


(a) 


* y 

X X 


(b) 


x 0 

y y 


(c) 


* y 
y x 


Discussion 

Discovery 

2o. 

(a) If A is an w x n matrix and if b is an n x 1 matrix, what conditions would you impose to ensure 
that the equation x = Ax 4- b has a unique solution for x? 


(b) Assuming that your conditions are satisfied, find a formula for the solution in terms of an 
appropriate inverse. 


Suppose that A is an invertible n x n matrix. Must the system of equations Ax = x have a unique 
29. solution? Explain your reasoning. 

Is it possible to have AB = 1 without B being the inverse of A? Explain your reasoning. 

30 

Create a theorem by rewriting Theorem 1.6.5 in contrapositive form (see Exercise 34 of Section 1.4). 

31. 
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1.7 

DIAGONAL, 
TRIANGULAR, AND 
SYMMETRIC MATRICES 


In this section we shall consider certain classes of matrices that have special 
forms. The matrices that we study in this section are among the most 
important kinds of matrices encountered in linear algebra and will arise in 
many different settings throughout the text. 


Diagonal Matrices 


A square matrix in which all the entries off the main diagonal are zero is called a diagonal matrix. Here are some examples: 


2 0 
0 -5 


1 0 0 
0 1 0 
0 0 1 


6 0 0 0 

0-400 
0 0 0 0 

0 0 0 8 


A general n xn diagonal matrix D can be written as 

'^1 0 . 0 

D= 0 d 2 - 0 

0 0 - d n 


( 1 ) 


A diagonal matrix is invertible if and only if all of its diagonal entries are nonzero; in this case the inverse of 1 is 


1/ofl 0 . 0 

0 \id 2 - 0 


0 0 - 1 id n 


The reader should verify that = D~^D = /• 


Powers of diagonal matrices are easy to compute; we leave it for the reader to verify that if D is the diagonal matrix 1 and k is a 
positive integer, then 


D k 


df 0 - 0 

0 d\ - 0 


0 



Inverses and Powers of Diagonal Matrices 


If 



0 0 
-3 0 
0 2 


then 



















II 

1 1 

1 

1 0 0 

0 -I 0 

3 

ii 

'1 0 o' 

0 -243 0 

II 

1 

1 0 0 

0 --L- o 

243 


0 0 I 

2 


0 0 32 


0 0 

32 


Matrix products that involve diagonal factors are especially easy to compute. For example, 


'di 

0 

0 ' 

'an 

3 12 

an 

ai 4 ‘ 


d\a it 

d\a\2 

d\a 13 

d\a\4 

0 

d'i 

0 

«21 

3 22 

3 23 

a 24 

= 

^2 3 21 

d'ja -22 

^2 3 23 

d 2^24 

0 

0 

d 3 

a 3 i 

« 32 

a 33 

a 34 


^3 3 31 

d 3 a 3 2 

d 31333 

^3334 


an 

3 12 

an 

3 21 

a 22 

3 23 

3 31 

« 32 

<333 

a 41 

a 42 

a 43 


til 0 0 

0 d 2 0 
0 0 d 3 


d\a\\ d 3 a \2 ^3 fl 13 
d i<Z2i t3?2^22 ^3^23 
d ifl3i ^2 fl 32 ^3 fl 33 
d 1^41 ^2^42 diet 42 


♦ 


In words, to multiply a matrix A on the left by a diagonal matrix D, one can multiply successive rows of A by the successive 
diagonal entries ofD, and to multiply A on the right by D, one can multiply successive columns of A by the successive diagonal 
entries ofD. 


Triangular Matrices 

A square matrix in which all the entries above the main diagonal are zero is called lower triangular, and a square matrix in 
which all the entries below the main diagonal are zero is called upper triangular. A matrix that is either upper triangular or 
lower triangular is called triangular. 


Upper and Lower Triangular Matrices 


an 

«12 

an 

a i4 

0 

3 22 

3 23 

3 24 

0 

0 

a 33 

a 34 

0 

0 

0 

13 44 


t 

A general 4x4 upper 
triangular matrix 


an 

0 

0 

0 

3 21 

3 22 

0 

0 

a 3 i 

3 32 

a 33 

0 

a 41 

3 42 

a 43 

13 44 


T 

A general 4x4 lower 
triangular matrix 


♦ 


Remark Observe that diagonal matrices are both upper triangular and lower triangular since they have zeros below and above 
the main diagonal. Observe also that a square matrix in row-echelon form is upper triangular since it has zeros belowthe main 
diagonal. 


The following are four useful characterizations of triangular matrices. The reader will find it instructive to verify that the 
matrices in Example 2 have the stated properties. 


A square matrix A = [a,j ] is upper triangular if and only if the /th row starts with at j — 1 zeros. 


A square matrix A = [a,j ] is lower triangular if and only if the j th column starts with j — 1 zeros. 


























A square matrix A = [a,j] is upper triangular if and only if a,, = 0 for i > j. 


A square matrix A = [ctjj] is lower triangular if and only if = 0 for i < j. 


The following theorem lists some of the basic properties of triangular matrices. 


THEOREM 1.7.1 


(a) The transpose of a lower triangular matrix is upper triangular, and the transpose of an upper triangular matrix is 
lower triangular. 

(b) The product of lower triangular matrices is lower triangular, and the product of upper triangular matrices is upper 
triangular. 

(c) A triangular matrix is invertible if and only if its diagonal entries are all nonzero. 

(d) The inverse of an invertible lower triangular matrix is lower triangular, and the inverse of an invertible upper 
triangular matrix is upper triangular. 


Part (a) is evident from the fact that transposing a square matrix can be accomplished by reflecting the entries about the main 
diagonal; we omit the formal proof. We will prove ( b ), but we will defer the proofs of (c) and (d) to the next chapter, where we 
will have the tools to prove those results more efficiently. 


Proof (b) We will prove the result for lower triangular matrices; the proof for upper triangular matrices is similar. Let 
A = and B = [i,j] be lower triangular nxn matrices, and let C = [ c 3J - ] be the product C = AB • From the remark preceding 
this theorem, we can prove that C is lower triangular by showing that Cjj = 0 for j < j. But from the definition of matrix 
multiplication, 


Cjj — a i\b\j -+■ a i2&2j 'I f a iyfni 

If we assume that i < j, then the terms in this expression can be grouped as follows: 

Cjj = + fljAj + - + a;(/—!)%-!)/ + + + 

Terms in which the row Terms in which the row 

number of b is less than the number of a is less than the 

column number of b column number of a 

In the first grouping all of the b factors are zero since B is lower triangular, and in the second 
grouping all of the a factors are zero since A is lower triangular. Thus, = 0, which is what we 
wanted to prove. 






Upper Triangular Matrices 


Consider the upper triangular matrices 


'1 

3 

- f 


'3 

-2 

2' 

0 

2 

4 

B = 

0 

2 

- 1 

0 

0 

5 


0 

0 

1 


The matrix A is invertible, since its diagonal entries are nonzero, but the matrix B is not. We leave it for the reader to calculate 
the inverse of A by the method of Section 1.5 and show that 


1 


A -1 


0 


3 

2 

1 

2 


0 


0 


7 

5 

2 

5 

1 

5 


This inverse is upper triangular, as guaranteed by part ( d ) of Theorem 1.7.1. We also leave it for the reader to check that the 
product AB is 


AB = 


3 

0 

0 


-2 

0 

0 



This product is upper triangular, as guaranteed by part ( b ) of Theorem 1.7.1. 


Symmetric Matrices 

A square matrix A is called symmetric if A = A T - 


EXAMPLE 4 Symmetric Matrices 


The following matrices are symmetric, since each is equal to its own transpose (verify). 


7 -3 
-3 5 


1 4 5 
4-3 0 
3 0 7 


til 0 0 0 

0 d 2 o 0 
0 0 d 3 0 

0 0 0 d 4 


♦ 


It is easy to recognize symmetric matrices by inspection: The entries on the main diagonal may be arbitrary, but as shown in 
2,“mirror images” of entries across the main diagonal must be equal. 



( 2 ) 


This follows from the fact that transposing a square matrix can be accomplished by interchanging entries that are symmetrically 
positioned about the main diagonal. Expressed in terms of the individual entries, a matrix A = [«,■, ] is symmetric if and only if 
fljj = ctji for all values of i and j. As illustrated in Example 4, all diagonal matrices are symmetric. 


The following theorem lists the main algebraic properties of symmetric matrices. The proofs are direct consequences of 
Theorem 1.4.9 and are left for the reader. 





















THEOREM 1.7.2 


If A and B are symmetric matrices with the same size, and if k is any scalar, then: 

( a ) A 7 I s symmetric. 


(b) A B and A — B are symmetric. 


(c) fcA is symmetric. 


Remark It is not true, in general, that the product of symmetric matrices is symmetric. To see why this is so, let A and B be 
symmetric matrices with the same size. Then from part id) of Theorem 1.4.9 and the symmetry, we have 

(AB) T = B T A T = BA 

Since AS and BA are not usually equal, it follows that AS will not usually be symmetric. However, in the special case where 
AB = BA ,» the product AS will be symmetric. If A and B are matrices such that AB = BA* then we say that A and B commute. In 
summary: The product of two symmetric matrices is symmetric if and only if the matrices commute. 


Products of Symmetric Matrices 


The first of the following equations shows a product of symmetric matrices that is not symmetric, and the second shows a 
product of symmetric matrices that is symmetric. We conclude that the factors in the first equation do not commute, but those in 
the second equation do. We leave it for the reader to verify that this is so. 


'1 2' 

' -4 

f 


'-2 f 

_2 3_ 

1 

0_ 


_ — 5 2_ 


'1 2' 

'-4 3" 


'2 f 

2 3 

3 - 1 


1 3 


In general, a symmetric matrix need not be invertible; for example, a square zero matrix is symmetric, but not invertible. 
However, if a symmetric matrix is invertible, then that inverse is also symmetric. 


THEOREM 1.7.3 


If A is an invertible symmetric matrix, then A~^ is symmetric. 


Assume that A is symmetric and invertible. From Theorem 1.4.10 and the fact that A = A T , we have 

1 T T —1 1 

(^ _1 ) ={A t ) =A ~ 1 


















which proves that a - 1 is symmetric. 


Products AA r and A J A 

Matrix products of the form 7 and a ?a arise in a variety of applications. If A is an m x n matrix, then a T is an n x m matrix, 

so the products jyfl and a 7 A are both square matrices—the matrix jyfl has size m x and the matrix a 7 A has si ze n x n- 

Such products are always symmetric since 

T T 7’7't’ 7 ’ 7- T 7’7’T' 7 ’ 

(AA 7 ) = (A 7 ) A t = AA t and (A 7 A) = A 7 (A 7 ) = A 7 A 


The Product of a Matrix and Its Transpose Is Symmetric 


Let A be the 2x3 matrix 


Then 


A = 


1 

3 


-2 4 

0 -5 



10 

-2 

-11 

-2 

4 

-8 

-11 

-8 

41 

21 

- 17' 


- 17 

34 



Observe that A 7 A and AA 7 are symmetric as expected. 


♦ 


Later in this text, we will obtain general conditions on A under which AA 7 and A 7 A are invertible. However, in the special case 
where A is square, we have the following result. 


THEOREM 1.7.4 


If A is an invertible matrix, then AA 7 an d A 7 A are a ^ so invertible. 


Proof Since A is invertible, so is a 7 by Theorem 1.4.10. Thus AA 7 and a Ta are invertible, since they are the products of 
invertible matrices. 


Exercise Set 1.7 





















Determine whether the matrix is invertible; if so, find the inverse by inspection. 


1 . 


(a) 


2 0 
0 — 5 


(b) 


4 0 0 
0 0 0 
0 0 5 


(c) 


-10 0 
0 2 0 
0 0 I 

3 


Compute the product by inspection. 

2 . 


(a) 


(b) 


'3 

0 

O' 

2 

f 

0 

- 1 

0 

-4 

1 

0 

0 

2 

2 

5 


'2 

0 

0" 

4 

- 1 

0 

- 1 

0 

1 

2 

0 

0 

4 

-5 

1 



-3 0 0 
0 5 0 
0 0 2 


Find j 4 2 , and a — k by inspection. 

3. 


(a) A = 


1 0 
0 -2 


(b) I 0 0 
2 

A= 0 I 0 

0 0 I 

4 


Which of the following matrices are symmetric? 

4 . 


2 - 1 
1 2 


(a) 










(b) 


3 4 

4 0 


(c) 



-1 3 
5 1 
1 7 


(d) 


0 0 1 
0 2 0 
3 0 0 


By inspection, determine whether the given triangular matrix is invertible. 

5. 


(a) 


-12 4 
0 3 0 
0 0 5 


(b) 


0 1-25 
0 1 5 6 

0 0-31 
0 0 0 5 


6 . 


Find all values of a, b, and c for which A is symmetric. 


A = 


2 a — 2b I 2c 2a 4 b I c 

3 5 a A c 

0-2 7 


Find all values of a and b for which A and B are both not invertible. 

7. 

5 0 

0 2a-3b-l_ 

Use the given equation to determine by inspection whether the matrices on the left commute. 


A = 


a 4 b — 1 0 
0 3 


5 = 


(a) 

1 -3' 

'4 r 



_ — 3 2_ 

.i 2 . 

- 

(b) 

2-1' 

'3 2' 

r 


_ -1 3_ 

2 1_ 



8 . 







Show that A and B commute if a — d = lb- 


9 . 



B 


a b 
b d 


Find a diagonal matrix A that satisfies 

10 . 


(a) 100 

A 5 = 0-1 0 

0 0-1 


(b) 9 0 0 

A~ 2 = 0 4 0 
0 0 1 


11 . 

(a) Factor A into the form A = BD, where D is a diagonal matrix. 


A = 


3a n 

3*21 

3fl3i 


5*12 7a 13 
5 a 22 7 a 23 
5(332 7a22 


(b) Is your factorization the only one possible? Explain. 


Verify Theorem 1.7.1b for the product AB, where 


' -1 

2 

5' 


'2 

-8 

O' 

0 

1 

3 

, B = 

0 

2 

1 

0 

0 

-4 


0 

0 

3 


Verify Theorem 1.7.1<i for the matrices A and B in Exercise 12. 

13. 


Verify Theorem 1.7.3 for the given matrix A. 

14. 



15 . 


Let A be an « x « symmetric matrix. 






(a) Show that a 2 is symmetric. 


(b) Show that 2A 2 — 3A I /is symmetric. 


Let A be an n x n symmetric matrix. 

16 . 

(a) Show that is symmetric if k is any nonnegative integer. 

(b) If p(x) is a polynomial, is p(A) necessarily symmetric? Explain. 

Let A be an nxn upper triangular matrix, and let p(x) be a polynomial. Is p(A') necessarily upper triangular? Explain. 


Prove: If A T A = A> then A is symmetric and a = A 2 - 

18 . 

Find all 3 x3 diagonal matrices A that satisfy a 2 — 3A — 41 = 0- 

19 

Let A = [a,-,-] be an « x « matrix. Determine whether A is symmetric. 

20 . 

<a) a^ = i 2 A j 2 

(b) a ^ = i l -j 2 

(c) Ay = 2 i 4 2 j 

(d) Ay = 2i 2 4 2 j 2 

On the basis of your experience with 20, devise a general test that can be applied to a formula for fly to determine whether 
21* A = [fly] is symmetric. 

A square matrix A is called skew-symmetric if A T = - A- Prove: 

22 . 

(a) If A is an invertible skew-symmetric matrix, then a ~ 1 i s skew-symmetric. 



(b) If A and B are skew-symmetric, then so are a t , A B, A — B, and kA for any scalar k. 


(c) Every square matrix A can be expressed as the sum of a symmetric matrix and a skew-symmetric matrix. 


Hint Note the identity A = ^ (A 4- A T ) 4 ^ (A — A T ). 


We showed in the text that the product of symmetric matrices is symmetric if and only if the matrices commute. Is the 

23. product of commuting skew-symmetric matrices skew-symmetric? Explain. 

Note See Exercise 22 for terminology. 

If the matrix A can be expressed as A = L U, where L is a lower triangular matrix and U is an upper triangular matrix, 

24. then the linear system Ax = b can be expressed as £ Ux = b and can be solved in two steps: 


Step 1 . Let Ux = y, so that L Ux = b can be expressed as Ly = b. Solve this system. 


Step 2. Solve the system Ux = y for x. 


In each part, use this two-step method to solve the given system. 


-1 

o 

o 

'2 -13' 

'*f 


'1 

-2 3 0 

0 1 2 

*2 

= 

-2 

2 4 1 

0 0 4 

*3 


0 


(b) 


-1 

[■■O 

o 

o 

'3 -5 2' 


'*f 


'4 

4 1 0 

0 4 1 


*2 

= 

— 5 

-3 -2 3 

0 0 2 


*3 


2 


25. 


Find an upper triangular matrix that satisfies 


A 3 


1 30 

0 -8 


Discussion 

Discovery 


What is the maximum number of distinct entries that an n x n symmetric matrix can have? 
26. Explain your reasoning. 


Invent and prove a theorem that describes how to multiply two diagonal matrices. 

27. 


Suppose that A is a square matrix and D is a diagonal matrix such that AD = l What can you say 
28. about the matrix A? Explain your reasoning. 






















29 . 


(a) Make up a consistent linear system of five equations in five unknowns that has a lower 
triangular coefficient matrix with no zeros on or below the main diagonal. 

(b) Devise an efficient procedure for solving your system by hand. 

(c) Invent an appropriate name for your procedure. 


Indicate whether the statement is always true or sometimes false. Justify each answer. 

30. 

(a) If Ad 7 " is singular, then so is A. 


(b) If A B is symmetric, then so are A and B. 

(c) If A is an n x n matrix and Ax = p has only the trivial solution, then so does A T x = 0- 


(d) If a 2 is symmetric, then so is A. 
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Chapter 1 


Supplementary Exercises 


1. 


Use Gauss-Jordan elimination to solve for x 1 and y' in terms of x and y. 


x = fx' 


y 


7 = jx* 4- f/ 


Use Gauss-Jordan elimination to solve for x 1 and y f in terms of x and y. 

x = x ! cos 9 — y ! sm9 
y — x ! sm9-\- y ! c os 9 


Find a homogeneous linear system with two equations that are not multiples of one another and such that 


and 

are solutions of the system. 


xi = 1, xo.—— 1, X 3 = l, xa—2 

xi = 2, X'i = 0, X3 = 3, X4=—1 


A box containing pennies, nickels, and dimes has 13 coins with a total value of 83 cents. How many coins of each type 
4. are in the box? 


Find positive integers that satisfy 
x+ 7 + z = 9 
x + 5y 4 1 Oz = 44 


For which value(s) of a does the following system have zero solutions? One solution? Infinitely many solutions? 

xi f X 2 4- X 3 = 4 

X3 = 2 

(a 2 — 4 )x3 = a — 2 
Let 

a 0 b 2 
a a 4 4 
0 a 2 b 

be the augmented matrix for a linear system. Find for what values of a and b the system has 


(a) a unique solution. 


(b) a one-parameter solution. 





(c) a two-parameter solution. 


(d) no solution. 


Solve for x, y, and z- 
xy - ‘Z'fy ¥3zy = S 
2xy - 3 \[y ¥ 2zy = 7 
-xv + \fy -¥ 2zy = 4 


9. 


Find a matrix K such that AKB — C given that 



2 
0 1 



C = 


8 6-6 

6-1 1 
4 0 0 


How should the coefficients a, b, and c be chosen so that the system 

ax + by — 3z= —3 
— 2 x — by I cz = — 1 
ax I 3y — cz = — 3 

has the solution x — 1 , y — — 1 , and z-2p- 


In each part, solve the matrix equation for X. 

11 . 


(a) -10 1 

jr ii o 

3 1-1 


1 2 0 
-3 1 5 


(b) 


'1 

-1 2' 


' -5 

-1 O' 

3 

0 1 _ 


6 

-3 7_ 


(c) 

3 r 

X-X 

'1 4' 


-1 2 . 


2 0_ 


2 -2 
5 4 


12 . 


(a) Express the equations 


^1 =*1 -*2 I x 3 
yj = 3xi 4 *2 -4x3 
yz = ~ 2xi - 2x2 I 3x3 


and 


*i =4yi -y 2 i yz 

z 2 = -3y 1 4 5^2-73 



in the matrix forms Y=AX and z = BY. Then use these to obtain a direct relationship 
Z = CX between Z and X. 

(b) Use the equation z — CX obtained in (a) to express z\ and zi in terms of xj, xj, and 7:3. 


(c) Check the result in (b) by directly substituting the equations for y j, and y^ into the equations for z\ and z-i 
and then simplifying. 


If A is m x n and B is « x p, how many multiplication operations and how many addition operations are needed to 

13. calculate the matrix product AB‘? 

Let A be a square matrix. 

14. 

(a) Show that (I — A)~ l = I \- A 4 A 2 + A 3 if A 4 = 0- 

(b) Show that (I - A ) -1 = I I A + A 2 H-h A” if A” +1 = 0- 

Find values of a, b, and c such that the graph of the polynomial p{x) — ax 1 -\ bx c passes through the points (1, 2), 
15 ' (-1, 6), and (2, 3). 

16. (For Readers Who Have Studied Calculus) Find values of a , b, and c such that the graph of the polynomial 
p(x') = ax 2 | bx + c passes through the point (-1, 0) and has a horizontal tangent at (2, -9). 

Let j be the n x n matrix each of whose entries is 1. Show that if n > ], then 

17 ” 

Show that if a square matrix A satisfies A 3 + A A 3 — 2 A 11 = CK then so does jiJ . 

18. 

Prove: If B is invertible, then AB~^ = A if an d only if AB — BA- 

19. 

Prove: If A is invertible, then A F B and l 4 BA~^ are both invertible or both not invertible. 

20 . 

Prove that if A and B are w x « matrices, then 

21 . 

(a) tr(A 4 B) =tr(A) ftr(5) 


(b) tr(A:A) = A:tr(A) 



(c) tr(j4 7 ') = tr(y4) 


(d) tr(A3) =tr(5,4) 


22 . 


Use Exercise 21 to show that there are no square matrices A and B such that 

AB-BA= 1 


23. 


Prove: If A is an m x n matrix and B is the n x 1 matrix each of whose entries is 1 / n , then 


AB = 


n 

n 


where F is the average of the entries in the /th row of A. 


24. (For Readers Who Have Studied Calculus) If the entries of the matrix 


C = 


cnCO 

^ 21 C^) 


C120) 

c 22 00 




are differentiable functions of x, then we define 


dC 

dx 


cute) 

c 2lW c 22 (^) 

iiOO 42w 


ci H (x) 
c 2n 00 

c mn 00 

c !«C x ) 

4h<*) 

c mn 0 ) 


Show that if the entries in A and B are differentiable functions of x and the sizes of the 
matrices are such that the stated operations can be performed, then 


(a) J-(kA)=k^ 

dx dx 


(b) -j-(a 4 B) = iA + dIL 
dx dx dx 


(c) = 4Ab + A$f- 

dx dx dx 


25. (For Readers Who Have Studied Calculus) Use part (c) of Exercise 24 to show that 

gLA ^—1 dA ^—1 

dx dx 

State all the assumptions you make in obtaining this formula. 



Find the values of a, b, and c that will make the equation 

x 2 I x — 2 _ a | bx I c 

(3jtr — 1)(jr 2 F 1) 3 * “ 1 i 3 4 1 

an identity. 

Hint Multiply through by (3x — 1) {x 2 +1) and equate the corresponding coefficients of the polynomials on each side 
of the resulting equation. 


If P is an n x 1 matrix such that p T p = 1 , then H — I — 2PP T ^ called the corresponding Householder matrix (named 
after the American mathematician A. S. Householder). 

(a) Verify that p T p = \ if P T — [f ^ 4 ^ an d compute the corresponding Householder matrix. 


(b) Prove that if H is any Householder matrix, then ft _ ftT and ftjft = /• 


(c) Verify that the Householder matrix found in part (a) satisfies the conditions proved in part (b). 


Assuming that the stated inverses exist, prove the following equalities. 

28. 

(a) (C -1 4 D -1 ) -1 = C(C4 D)~ l D 

(b) (1 4 CD ) _1 C = C(1 4 DC ) -1 

(c) (C f DD r )~ l D=C~ l D(I+ D T C~ 1 D)~ 1 


29. 

(a) Show that if a =* *, then 


a n ~ l b + a n - 2 b 2 F- 


I ab”- l +b n 


M + l _ ^M + l 
a — b 


(b) Use the result in part (a) to find A” if 


A = 


a 0 0 
0*0 
1 0 c 


Note This exercise is based on a problem by John M. Johnson, The Mathematics Teacher , Vol. 85, No. 9, 1992. 
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Chapter 1 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Section 1.1 

Tl. Numbers and Numerical Operations Read your documentation on entering and displaying numbers and performing the 
basic arithmetic operations of addition, subtraction, multiplication, division, raising numbers to powers, and extraction of 
roots. Deter mi ne how to control the number of digits in the screen display of a decimal number. If you are using a CAS, in 
which case you can compute with exact numbers rather than decimal approximations, then learn how to enter such numbers 
as ;r, ^2, and j exactly and convert them to decimal form. Experiment with numbers of your own choosing until you feel you 
have mastered the procedures and operations. 


Section 1.2 

Tl. Matrices and Reduced Row-Echelon Form Read your documentation on how to enter matrices and how to find the 

reduced row-echelon form of a matrix. Then use your utility to find the reduced row-echelon form of the augmented matrix in 
Example 4 of Section 1.2. 


T2. Linear Systems With a Unique Solution Read your documentation on how to solve a linear system, and then use your 
utility to solve the linear system in Example 3 of Section 1.1. Also, solve the system by reducing the augmented matrix to 
reduced row-echelon form. 


T3. Linear Systems With Infinitely Many Solutions Technology utilities vary on how they handle linear systems with infinitely 
many solutions. See how your utility handles the system in Example 4 of Section 1.2. 


T4. Inconsistent Linear Systems Technology utilities will often successfully identify inconsistent linear systems, but they can 
sometimes be fooled into reporting an inconsistent system as consistent, or vice versa. This typically happens when some of 
the numbers that occur in the computations are so small that roundoff error makes it difficult for the utility to determine 
whether or not they are equal to zero. Create some inconsistent linear systems and see how your utility handles them. 

A polynomial whose graph passes through a given set of points is called an interpolating polynomial for those points. Some 
T5. technology utilities have specific commands for finding interpolating polynomials. If your utility has this capability, read the 
documentation and then use this feature to solve Exercise 25 of Section 1.2. 


Section 1.3 



Tl. Matrix Operations Read your documentation on how to perform the basic operations on matrices—addition, subtraction, 
multiplication by scalars, and multiplication of matrices. Then perform the computations in Examples Example 3, Example 4, 
and Example 5. See what happens when you try to perform an operation on matrices with inconsistent sizes. 


T2. 


Evaluate the expression _ 3 A 3 4 . 7 ^ _ 4 / for the matrix 


A = 


1 -2 
-4 5 

7 -8 


3 

-6 

9 


T3. Extracting Rows and Columns Read your documentation on how to extract rows and columns from a matrix, and then use 
your utility to extract various rows and columns from a matrix of your choice. 


T4. Transpose and Trace Read your documentation on how to find the transpose and trace of a matrix, and then use your utility 
to find the transpose of the matrix A in Formula (12) and the trace of the matrix B in Example 12. 


T5. Constructing an Augmented Matrix Read your documentation on how to create an augmented matrix [j4|b] from matrices 
A and b that have previously been entered. Then use your utility to form the augmented matrix for the system ^ = b in 
Example 4 of Section 1.1 from the matrices A and b. 


Section 1.4 


Tl. Zero and Identity Matrices Typing in entries of a matrix can be tedious, so many technology utilities provide shortcuts for 
entering zero and identity matrices. Read your documentation on how to do this, and then enter some zero and identity 
matrices of various sizes. 


T2. Inverse Read your documentation on how to find the inverse of a matrix, and then use your utility to perform the 
computations in Example 7. 


T3. Formula for the Inverse If you are working with a CAS, use it to confirm Theorem 1.4.5. 


T4. Powers of a Matrix Read your documentation on how to find powers of a matrix, and then use your utility to find various 
positive and negative powers of the matrix A in Example 8 . 


Let 


T5. 


A = 


1 1 I 

2 3 


I 1 I 

4 5 

I 1 1 

6 7 


Describe what happens to the matrix w hen k is allowed to increase indefinitely (that is, as k —, 00 )• 







By experimenting with different values of n, find an expression for the inverse of an nxn matrix of the form 


1 

2 

3 

4 ■ 

■■ ft — 1 

ft 

0 

1 

2 

3 ■ 

ft — 2 

ft — 1 

0 

0 

1 

2 ■ 

■■ ft — 3 

ft — 2 

0 

0 

0 

0 ■ 

■■ 1 

2 

0 

0 

0 

0 ■ 

■■ 0 

1 


Section 1.5 

Use your technology utility to verify Theorem 1.5.1 in several specific cases. 

Tl. 


T2. Singular Matrices Find the inverse of the matrix in Example 4, and then see what your utility does when you try to invert 
the matrix in Example 5. 


Section 1.6 


Tl. 


T2. 

T3. 


Solving j4 x = b by Inversion Use the method of Example 4 to solve the system in Example 3 of Section 1.1. 

Compare the solution of ^x = v, by Gaussian elimination and by inversion for several large matrices. Can you see the 
superiority of the former approach? 

Solve the linear system = 2x> given that 


A = 


0 0 
1 2 
1 0 


Section 1.7 

Tl. Diagonal, Symmetric, and Triangular Matrices Many technology utilities provide short-cuts for entering diagonal, 

symmetric, and triangular matrices. Read your documentation on how to do this, and then experiment with entering various 
matrices of these types. 


T2. Properties of Triangular Matrices Confirm the results in Theorem 1.7.1 using some triangular matrices of your choice. 

Confirm the results in Theorem 1.7.4. What happens if A is not square? 

T3. 
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CHAPTER 


Determinants 


NTRODUCTlOh We are all familiar with functions such as f( x ) = sinx and /OO =x 2 , which associate a real number 
f (x) with a real value of the variable *. Since both x and j (*) assume only real values, such functions are described as 
real-valued functions of a real variable. In this section we shall study the “determinant function,” which is a real-valued 
function of a matrix variable in the sense that it associates a real number f ( x ) with a square matrix x■ Our work on 
determinant functions will have important applications to the theory of systems of linear equations and will also lead us to an 
explicit formula for the inverse of an invertible matrix. 


Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 





2 C As noted in the introduction to this chapter, a “determinant” is a certain kind 

■ 1 of function that associates a real number with a square matrix. In this 

DETERMINANTS BY section we will define this function. /Is a consequence of our work here, we 

COFACTOR EXPANSION w '^ °^ ain a f° rmu l a tor the inverse of an invertible matrix as well as a 

formula for the solution to certain systems of linear equations in terms of 
determinants. 


Recall from Theorem 1.4.5 that the 2x2 matrix 



b 

d 


is invertible if ad — be * 0- The expression ad — be occurs so frequently in mathematics that it has a name; it is called the 
determinant of the matrix A and is denoted by the symbol det(/T) or \A\. With this notation, the formula for a~^ given in 
Theorem 1.4.5 is 

a -1 1 d —b 

~ det(j4) [-c a_ 

One of the goals of this chapter is to obtain analogs of this formula to square matrices of higher order. This will require that 
we extend the concept of a determinant to square matrices of all orders. 


Minors and Cofactors 

There are several ways in which we might proceed. The approach in this section is a recursive approach: It defines the 
determinant of an n x « matrix in terms of the determinants of certain (« — l)x(« — 1) matrices. The (« — 1) x (« — 1) 
matrices that will appear in this definition are submatrices of the original matrix. These submatrices are given a special 
name: 


DEFINITION 


If A is a square matrix, then the minor of entry ctij is denoted by and is defined to be the determinant of the 
submatrix that remains after the jth row and jth column are deleted from A- The number ( _ 1) ! +jl is denoted by Cy 
and is called the cofactor of entry &ij. 


Finding Minors and Cofactors 


Let 


A = 


3 1 -4 

2 5 6 

1 4 8 


The minor of entry a 11 is 

















The cofactor of a\\ is 
Similarly, the minor of entry a -$2 is 


The cofactor of 032 is 


M II - 


h 

4 


5 6 
4 H 


= 16 


Cu = (-l) 1+1 Mn = Mu = 16 


— 




C 32 = ( - 1 ) 3+2 M 32 = - M 32 = - 26 


♦ 


Note that the cofactor and the minor of an element differ only in sign; that is, C,y = ± A quick way to determine 
whether to use + or _ is to use the fact that the sign relating and is in the ;th row and j th column of the 
“checkerboard” array 


+ 

— 

4 - 

4 

— 

+ 

- 4 

— 

+ 

— 

+ - 

+ 

— 

4 

- 4 

— 


For example, Cn = M\\i C 21 = — Mi\i C 12 = — Myt C 22 = M 22 , an< ^ so on- 


Strictly speaking, the determinant of a matrix is a number. However, it is common practice to “abuse” the terminology 
slightly and use the term determinant to refer to the matrix whose determinant is being computed. Thus we might refer to 

3 1 

4 -2 

as a 2 x 2 determinant and call 3 the entry in the first row and first column of the determinant. 


Cofactor Expansions 

The definition of a 3 x 3 determinant in terms of minors and cofactors is 

det(j 4 ) H ^i 2 ( — ^12) + 

= «llC'li+312^12+013^13 ^ 

Equation 1 shows that the determinant of A can be computed by multiplying the entries in the first row of A by their 
corresponding cofactors and adding the resulting products. More generally, we define the determinant of an n x n matrix to 
be 

det(j 4 ) =l 3 iiCh 4 12^ 12 H- 1 - n 

This method of evaluating det(j4) is called cofactor expansion along the first row of A- 


Cofactor Expansion Along the First Row 




















Let A — 


. Evaluate det(^4) by cofactor expansion along the first row of A- 


3 1 0 

-2-4 3 

5 4-2 


Solution 

From 1, 



3 

1 

0 


det(j4) = 

-2 

-4 

3 

= 3 


5 

4 

-2 



-2 

5 



4 0 


-2 

5 


= 3( —4) — (1)( — ll) + 0= —1 



If A is a 3 x 3 matrix, then its determinant is 


det(j4) = 


<311 a\2 a 13 
a 2\ a 22 a 23 
a 3 l 33 2 a 33 


= ail 


«22 «23 
332 333 


-«12 


«21 

331 


«23 

333 


+ «13 


«21 

331 


a 22 

a 32 


= a 11022«33 23^32) “ <*12 021^33 “ «23«3l) + <313(^21^32 —^22^31) 


♦ 


( 2 ) 


= 3 11322^33 4 <312<323<331 4 313<321<332 — <313<322<331 — <312<321<333 — 311<323<332 


( 3 ) 


By rearranging the terms in 3 in various ways, it is possible to obtain other formulas like 2. There should be no trouble 
checking that all of the following are correct (see Exercise 28): 

det(j4) = a\\C\\ 4 < 212^12 H < 213^13 
= 311^11 4 <221^21 4 <231(731 
— 321^21 4 < 222^22 4 ^ 23^23 

(Z 

= 312^12 4 0,22^22 I a 32^32 
= 33l(73i 4 <232(732 4 3 33^33 
= 3 13(^13 4 323C23 h <233(733 

Note that in each equation, the entries and cofactors all come from the same row or column. These equations are called the 
cofactor expansions of det( 4 l)- 

The results we have just given for 3 x 3 matrices form a special case of the following general theorem, which we state 
without proof. 


THEOREM 2.1.1 

Expansions by Cofactors 

The determinant of an nxn matrix A can t> e computed by multiplying the entries in any row (or column) by their 
cofactors and adding the resulting products; that is, for each 1 < i < n and 1 < j < n. 

det(j4) = a\jC\j 4 ajjCjj 4■■■ 4 3 H j( 

(cofactor expansion along the jth column) 

and 





















det(j4) = f aaCa 4 -4 a in C in 

[c of actor expansion along the ith row) 


Note that we may choose any row or any column. 


Cofactor Expansion Along the First Column 


Let A be the matrix in Example 2. Evaluate det(4l) by cofactor expansion along the first column of A- 


Solution 


From 4 


det(A) = 


3 1 

-2 -4 


0 
3 

4 -2 


= 3 


■4 3 

4 -2 


-(- 2 ) 


1 0 
4 -2 


+ 5 


1 0 
-4 3 


= 3( — 4) — (— 2)( — 2) 4 5(3) = — 1 


This agrees with the result obtained in Example 2. 


Remark In this example we had to compute three cofactors, but in Example 2 we only had to compute two of them, since 
the third was multiplied by zero. In general, the best strategy for evaluating a determinant by cofactor expansion is to expand 
along a row or column having the largest number of zeros. 


EXAI Smart Choice of Row or Column 


If A is the 4 x 4 matrix 


A = 


10 0-1 

3 1 2 2 

10-2 1 
2 0 0 1 


then to find det(2l) it will be easiest to use cofactor expansion along the second column, since it has the most zeros: 

1 0-1 

det(2l) = 1 - 1-2 1 

2 0 1 

For the 3x3 determinant, it will be easiest to use cofactor expansion along its second column, since it has the most zeros: 

det(j4) = 1 - - 2 ■ ! 

2 1 

= -2(142) 

= -6 

We would have found the same answer if we had used any other row or column. 
























Adjoint of a Matrix 


In a cofactor expansion we compute det(j 4 ) by multiplying the entries in a row or column by their cofactors and adding the 
resulting products. It turns out that if one multiplies the entries in any row by the corresponding cofactors from a different 
row, the sum of these products is always zero. (This result also holds for columns.) Although we omit the general proof, the 
next example illustrates the idea of the proof in a special case. 


Entries and Cofactors from Different Rows 


Let 


on 

a \2 

013 

«21 

a 22 

«23 

031 

a 32 

033 


Consider the quantity 


flllC3i -1- 012C32 4 013033 


that is formed by multiplying the entries in the first row by the cofactors of the corresponding entries in the third row and 
adding the resulting products. We now show that this quantity is equal to zero by the following trick. Construct a new matrix 
A! by replacing the third row of A with another copy of the first row. Thus 


A' = 


a U a \2 a 13 
«21 «22 «23 
an a 12 a 13 


Let Cg , Cj2’ C33 be the cofactors of the entries in the third row of A f . Since the first two rows of A and A ! are the same, and 
since the computations of C-^ C-^ > C32’ an( J C33 involve only entries from the first two rows of A and A'\ it 

follows that 

C3i = c5 lt c 32 = c' 2 , C33 = c5 3 

Since A f has two identical rows, it follows from 3 that 


det(j 4 ') = 0 

On the other hand, evaluating det(j 4 ,f ) by cofactor expansion along the third row gives 

det(-d') —auC^ 4 a 12^32 + ^^33 — a U ^31 4 a 12^32 4 ^ 13^33 


( 5 ) 


( 6 ) 


From 5 and 6 we obtain 


011031 4 ^ 12^32 4 ^13033 = 0 


Now we’ll use this fact to get a formula for ^-t. 


DEFINITION 


If A is any nx.n matrix and C,j is the cofactor of o,y, then the matrix 













c n 

c 12 ■ 

■■ c ln 

<01 

<02 ■ 

■■ c 2n 

c ni 

Ch2 ' 



is called the matrix of cofactors from A. The transpose of this matrix is called the adjoint of A and is denoted by a dj (A) • 


Adjoint of a 3 x 3 Matrix 


Let 


The cofactors of A are 


so the matrix of cofactors is 


and the adjoint of A is 


A = 


3 

1 

2 


2 - 1 
6 3 

-4 0 


C'u = 12 

Q 

to 

II 

1 

II 

o 

^21=4 

^22 = 2 

C 23 = 16 

C 31 = 12 

Q 

LO 

to 

II 

1 

o 

C 33 = 16 


12 

6 

- 16 


4 

2 

16 


12 

- 10 

16 



12 

4 

12 

1 = 

6 

2 - 

- 10 


- 16 

16 

16 


We are now in a position to derive a formula for the inverse of an invertible matrix. We need to use an important fact that 
will be proved in Section 2.3: The square matrix A is invertible if and only if det(^4) is not zero. 


THEOREM 2 . 1.2 


Inverse of a Matrix Using Its Adjoint 



If A is an invertible matrix, then 

^ 1= d c ,U adj(jl) 

( 7 ) 


Proof We show first that 


dadj(d) = det(d)/ 















Consider the product 


A adj (j4) = 


'an 

«12 

■" a \n 





a 21 

a 22 

■" a '2n 

'c n 

C 2 1 • 

Cjl • 

,a ^hI 

i 

\ 

= 

Cl2 

C 22 • 

'■ Cfl ■ 

- C h2 

an 

A'2 

"■ a in 

: 

: 

: 

a n 1 


a nn 

Cln 

C 2h ' 

" Cj'w ■ 

.. c 

-- HH 


The entry in the jth row and yth column of the product ^adjOl) is 


a ! 1 Cj 1 +■ "1“ I" 


( 8 ) 


(see the shaded lines above). 

If i = j, then 8 is the cofactor expansion of det(-d) along the ;th row of A (Theorem 2.1.1), and if i ^ j, then the a's and the 
cofactors come from different rows of A, so the value of 8 is zero. Therefore, 


A adj (j!) = 


det(j4) 0 
0 det(j4) 


0 


0 


0 

0 

det(j4) 


= det(j4)/ 


(9) 


Since A is invertible, det(j4) ^ 0- Therefore, Equation 9 can be rewritten as 


1 


det(j4) 

Multiplying both sides on the left by a -1 yields 


[j4adj(j4)] =/ or A 


det(j4) 


-adj(^) 


= 1 


A~ { = 


det(jl) 


-adj(^) 


Using the Adjoint to Find an Inverse Matrix 


Use 7 to find the inverse of the matrix A in Example 6. 


Solution 

The reader can check that det(j4) = 64- Thus 


A~ l = 


det(j4) 




12 

4 

12 

6 

2 

- 10 

16 

16 

16 


12 

4 

12 

64 

64 

64 

6 

2 

10 

64 

64 

64 

16 

16 

16 

64 

64 

64 


♦ 


Applications of Formula 7 

Although the method in the preceding example is reasonable for inverting 3x3 matrices by hand, the inversion algorithm 
discussed in Section 1.5 is more efficient for larger matrices. It should be kept in mind, however, that the method of Section 
1.5 is just a computational procedure, whereas Formula 7 is an actual formula for the inverse. As we shall now see, this 
formula is useful for deriving properties of the inverse. 

In Section 1.7 we stated two results about inverses without proof. 




















Theorem 1.7.1c: A triangular matrix is invertible if and only if its diagonal entries are all nonzero. 


Theorem \.lAd: The inverse of an invertible lower triangular matrix is lower triangular, and the inverse of an invertible 
upper triangular matrix is upper triangular. 

We will now prove these results using the adjoint formula for the inverse. We need a preliminary result. 


THEOREM 2.1.3 


If A is an K xn triangular matrix (upper triangular, lower triangular, or diagonal), then det(A) A the product of the 
entries on the main diagonal of the matrix; that is, det(j4) — 


For simplicity of notation, we will prove the result for a 4 x 4 lower triangular matrix 


AH 

0 

0 

0 

a 2 \ 

22 

0 

0 

<331 

<332 

<333 

0 

<341 

<342 

<343 

<344 


The argument in the « x n case is similar, as is the case of upper triangular matrices. 


Proof of Theorem 2.1.3 (Ax A lower triangular case) By Theorem 2.1.1, the determinant of A may be found by cofactor 
expansion along the first row: 


det(2l) 


<311 

0 

0 

0 

<321 

<322 

0 

0 

<331 

«32 

<333 

0 

<341 

<342 

<343 

<344 


<*22 


= 11 


a 32 
<342 


0 0 
<333 0 

1343 <344 


Once again, it's easy to expand along the first row: 


det(2l) =fliii322 


<333 
a 43 


0 

a 44 


= < 3 i 1<322<333I<344I 
= < 3 i 1<322<333<344 


where we have used the convention that the determinant of a 1 x 1 matrix [<3] is a - 


Determinant of an Upper Triangular Matrix 













— (2) ( — 3) (6) (9) (4) = —1296 


2 7-333 

0 -3 7 5 1 

0 0 6 7 6 

0 0 0 9 8 

0 0 0 0 4 


Proof of Theorem 1.7.1 c Let A = [a^ ] be a triangular matrix, so that its diagonal entries are 

^ll- a 22> -■-> a Ytn 

From Theorem 2.1.3, the matrix a is invertible if and only if 

det(-4) = awajr-aym 

is nonzero, which is true if and only if the diagonal entries are all nonzero. 


We leave it as an exercise for the reader to use the adjoint formula for a~ 1 to show that if A = is an invertible 
triangular matrix, then the successive diagonal entries of A~ l are 


J_ l_ _L_ 

a n ’ <322 

(See Example 3 of Section 1.7.) 


Proof of Theorem 1.7. Id We will prove the result for upper triangular matrices and leave the lower triangular case as an 
exercise. Assume that A is upper triangular and invertible. Since 


A~ [ 


1 

det(j4) 


adj(^) 


we can prove that a~ 1 is upper triangular by showing that adj(^) is upper triangular, or, 
equivalently, that the matrix of cofactors is lower triangular. We can do this by showing that every 
cofactor Cj, with i< j (i.e., above the main diagonal) is zero. Since 

c,, = (-i)' + 4/,. 


it suffices to show that each minor with j< j is zero. For this purpose, let B }] be the matrix that 
results when the jth row and jth column of a are deleted, so 


Mi] - det (Bij) 


( 10 ) 


From the assumption that i<j, it follows that is upper triangular (Exercise 32). Since A is upper 
triangular, its (? + i)-st row begins with at least j zeros. But the /'th row of B^ is the (? + i)-st row of 
A with the entry in the jth column removed. Since i< j, none of the first j zeros is removed by 
deleting the jth column; thus the /'th row of B^ starts with at least i zeros, which implies that this 
row has a zero on the main diagonal. It now follows from Theorem 2.1.3 that det^) = 0 and from 
10 that Mij = 0 . 


Cramer's Rule 

The next theorem provides a formula for the solution of certain linear systems of « equations in n unknowns. This formula, 
known as Cramer's rule , is of marginal interest for computational purposes, but it is useful for studying the mathematical 
properties of a solution without the need for solving the system. 








THEOREM 2.1.4 


Cramer's Rule 

If Ax = b is a system off linear equations in « unknowns such that det(A) ^ 0. then the system has a unique solution. 
This solution is 


__ = detUi) det(A 2 ) _ <*et(A„) 

1 det(A) ’ 2 det (A) ’ ’ M det(A) 


where Aj is the matrix obtained by replacing the entries in the yth column of A by the entries in 
the matrix 





Proof If det(A) * 0, then A is invertible, and by Theorem 1.6.2, x = A~ l b is the unique solution of Ax = b- Therefore, by 
Theorem 2.1.2 we have 




1 

det(A) 


Cu 

C 2 1 ■ 

'■ C„1 

h 

^12 

C*22 ' 

■■ c h2 

h 

Ci» 

c 2h ■ 


b n 


Multiplying the matrices out gives 


det (A) 


AlC-n 


i?lC i H 


*2^21 ”1 A bnCn i 

*2^22 H-1 ^rPrO. 

^2^2m ”1 f ^rPrm 


The entry in the jth row of x is therefore 


Now let 



i 


& 2 C 2 y A 

“A 






det (A) 





a n 

a Yl 

- oij-1 

*1 

^lj'Ai ‘ 

" a ln 

A j = 

<*21 

®22 

- a 2j-l 

h 

a 2j+\ ■ 

,J a 2n 



a n2 

a nj — 1 


^HJ+l ‘ 



( 11 ) 


Since Aj differs from A only in the yth column, it follows that the cofactors of entries ---, b n in A, are the same as the 

cofactors of the corresponding entries in the /th column of A- The cofactor expansion of det (Ay) along the /th column is 
therefore 


dct(Ay) —-T ^2^2j A '"A 

Substituting this result in 11 gives 

det(A f ) 

Xj= det (A) 





















Using Cramer's Rule to Solve a Linear System 


Use Cramer's rule to solve 

*1 + 4- 2*3 — 6 

— 3*1 4- 4x2 4- 6 x 3 = 30 
— *1 — 2*2 4- 3x2 = 3 


Solution 


Therefore, 



1 0 2' 


' 6 0 2' 

A = 

-3 4 6 

-1 -2 3 

^1 = 

30 4 6 

3-2 3 



1 

6 

2 


i 

0 

6 

41 2 = 

-3 

30 

6 

A3 = 

-3 

4 

30 


- 1 

8 

3 


- 1 

-2 

8 


det(j4i) 

-40 

- 10 

det(j4 2 ) 

72 

18 

1 det(j4) 

44 

11 * 

2 det(j4) 

44 " 

11 

_ det(j4 3 ) 

3 det(j4) 

152 
' 44 

33 

11 






Gabriel Cramer (1704-1752) was a Swiss mathematician. Although Cramer does not rank with the great mathematicians 
of his time, his contributions as a disseminator of mathematical ideas have earned him a well-deserved place in the history 
of mathematics. Cramer traveled extensively and met many of the leading mathematicians of his day. 


Cramer's most widely known work, Introduction a Vanalyse des lignes courbes algebriques (1750), was a study and 
classification of algebraic curves; Cramer's rule appeared in the appendix. Although the rule bears his name, variations of 
the idea were formulated earlier by various mathematicians. However, Cramer's superior notation helped clarify and 


















popularize the technique. 


Overwork combined with a fall from a carriage led to his death at the age of 48. Cramer was apparently a good-natured 
and pleasant person with broad interests. He wrote on philosophy of law and government and the history of mathematics. 
He served in public office, participated in artillery and fortifications activities for the government, instructed workers on 
techniques of cathedral repair, and undertook excavations of cathedral archives. Cramer received numerous honors for his 
activities. 


Remark To solve a system of n equations in n unknowns by Cramer's rule, it is necessary to evaluate n 4. 1 determinants of 
n x n 

matrices. For systems with more than three equations, Gaussian elimination is far more efficient. However, Cramer's rule 
does give a formula for the solution if the determinant of the coefficient matrix is nonzero. 


Exercise Set 2.1 


Click here for Just Ask! 


Let 


1 . 


A = 


1 -2 

6 

-3 


7 - 1 
1 4 


(a) Find all the minors of A- 

(b) Find all the cofactors. 


2 . 


Let 






"4 

— 

1 

1 

A _ 

0 


0 - 

-3 

Jl - 

4 


1 

0 


4 


1 

3 

Find 





(a) 

M 13 

and 

C 13 

(b) 

M 23 

and 

C73 

(c) 

M22 

and 

^22 

(d) 

M 2 \ 

and 

C 2 l 








Evaluate the determinant of the matrix in Exercise 1 by a cofactor expansion along 

3 . 

(a) the first row 

(b) the first column 

(c) the second row 

(d) the second column 

(e) the third row 

(f) the third column 


For the matrix in Exercise 1, find 

4. 


(a) adj (j4) 


(b) ^-1 using Theorem 2.1.2 


In Exercises 5-10 evaluate det(j4) by a cofactor expansion along a row or column of your choice. 


5. A = 


-3 0 7 
2 5 1 
-10 5 


6 . A = 


3 1 

0 -4 
-3 5 



1 k k 2 
1 k k 2 
1 k k 2 


8 - ,4 = 


k + 1 
2 
5 


k- 1 
k-3 
k I 1 


7 

4 

£ 



3 

2 

1 

10 







10 . 


A = 


4 

0 

0 

1 

0 

3 

3 

3 

- 1 

0 

1 

2 

4 

2 

3 

9 

4 

6 

2 

3 

2 

2 

4 

2 

3 


In Exercises 11-14 find using Theorem 2.1.2. 


11. A = 


2 5 5 

-1 -10 
2 4 3 


12. A = 


2 0 3 

0 3 2 

-2 0-4 


13. A = 


-3 5 

1 -3 

0 2 


14. A = 


2 0 0 
8 1 0 
-5 3 6 


Let 

15. 


A = 


1 3 

2 5 
1 3 
1 3 


1 1 

2 2 
8 9 
2 2 


(a) Evaluate a -1 using Theorem 2.1.2. 


(b) Evaluate A ~ 1 using the method of Example 4 in Section 1.5. 


(c) Which method involves less computation? 

In Exercises 16-21 solve by Cramer's rule, where it applies. 

7xi — 2 xt — 3 
3xi f *2 = 5 

4x f 5y —2 
17 * llx I y \ 2z—3 
x f 5y 4 2z = 1 

x - 4y + z = 6 

18 - 4x- y-i 2z= -1 

2x + 2y — 3z= — 20 




19 . 


20 . 


21 . 


*1 - 

•3x2 i 

x 3 = 


4 


2 xi - 

■ *2 


= - 

-2 


Ax 

1 

- 3x3 = 


0 


-*1 

-4x2 

4 2x3 

1 

x 4 = 

-32 

2 xi 

- x 2 

4 7x3 

1 

9 x 4 = 

14 

-*1 

+ *2 

4 3x3 

+ 

x 4 = 

11 

*1 

- 2X2 

4 *3 

— 

II 

'=f 

K 

-4 

3 xi 

- x 2 

1 x 3 

= 

4 


-*1 

4 7X2 

— 2x3 

= 

1 


2 xi 

4 6x2 

- *3 

= 

5 



Show that the matrix 


22 . 


A = 


cos 9 sin# 0 
— sin# cos# 0 
0 0 1 


23. 


24. 


25. 


is invertible for all values of #; then find A -1 using Theorem 2.1.2. 

Use Cramer's rule to solve for y without solving for x , z , and w- 
Ax + y \ z+ w = 6 

3x + 7y =- z-\ w = 1 

lx A- 3y — 5z f 8w = — 3 
x + y + z 2w= 3 

Let Ax — b be the system in Exercise 23. 

(a) Solve by Cramer's rule. 

(b) Solve by Gauss-Jordan elimination. 

(c) Which method involves fewer computations? 

Prove that if det(j4) = 1 and all the entries in A are integers, then all the entries in A -1 are integers. 


Let Ax = b be a system of n linear equations in n unknowns with integer coefficients and integer constants. Prove that if 
26. det(j4) = 1, the solution x has integer entries. 


27. 


Prove that if A is an invertible lower triangular matrix, then A -1 is lower triangular. 


Derive the last cofactor expansion listed in Formula 4. 


28 . 



29 . 


Prove: The equation of the line through the distinct points (a \,b\) and (a 2i A 2 ) can be written as 


* 7 1 

a\ 1 
ct'i b2 1 


= 0 


30. 


Prove: (xi, y 1 ), (* 2 ,7 2 )> an( f (*3-73) are collinear points if and only if 


*1 71 1 
*2 72 1 
*3 73 1 


= 0 


31. 


(a) 


If-3 = 


An 

An 

0 

A 22 


is an “upper triangular” block matrix, where and A 22 are square matrices, then 


det(A) = det^n) det(A 22 )- Use this result to evaluate det(A) for 


'2 

-1 

2 5 

6 

4 

3 

-1 3 

4 

0 


1 3 

5 

0 

i ) 

—2 6 

2 

0 

0 

3 5 

2 


(b) Verify your answer in part (a) by using a cofactor expansion to evaluate det(A)- 


Prove that if A is upper triangular and Bu is the matrix that results when the zth row and yth column of A are deleted, 

L 

then By is upper triangular if j < j. 


Discussion 

Discovery 


What is the maximum number of zeros that a 4 x 4 matrix can have without having a zero 
33. determinant? Explain your reasoning. 


Let A be a matrix of the form 

34. 


* 

* 

0 

0 

0 

* 

* 

0 

0 

0 

* 

* 

0 

0 

0 

* 

* 

* 

* 

* 


How many different values can you obtain for det(A) by substituting numerical values (not 
necessarily all the same) for the *'s? Explain your reasoning. 


Indicate whether the statement is always true or sometimes false. Justify your answer by givin 
35. a logical argument or a counterexample. 


( a ) A adj (A) is a diagonal matrix for every square matrix A- 


















(b) In theory, Cramer's rule can be used to solve any system of linear equations, although 
the amount of computation may be enormous. 

(c) If A is invertible, then ac y (^4) must also be invertible. 


(d) If A has a row of zeros, then so does adj(A)- 
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2.2 

EVALUATING 
DETERMINANTS BY ROW 
REDUCTION 


In this section we shall show that the determinant of a square matrix can be 
evaluated by reducing the matrix to row-echelon form. This method is important 
since it is the most computationally efficient way to find the determinant of a 
general matrix. 


A Basic Theorem 

We begin with a fundamental theorem that will lead us to an efficient procedure for evaluating the determinant of a matrix of any 
order n . 

THEOREM 2.2.1 


Let Abe a square matrix. If A has a row of zeros or a column of zeros, then det(A) — 0- 


Proof By Theorem 2.1.1, the determinant of A found by cofactor expansion along the row or column of all zeros is 

det(A) = 0-Ci + 0 -C 2 + - + 0 -C„ 

where C\, C H are the cofactors for that row or column. Hence det(A) is zero. 

Here is another useful theorem: 

THEOREM 2.2.2 

7" 1 

Let A be a square matrix. Then det(A) = det(A ). 


By Theorem 2.1.1, the determinant of A found by cofactor expansion along its first row is the same as the determinant of 
found by cofactor expansion along its first column. 


Remark Because of Theorem 2.2.2, nearly every theorem about determinants that contains the word row in its statement is also true 
when the word column is substituted for row. To prove a column statement, one need only transpose the matrix in question, to convert 
the column statement to a row statement, and then apply the corresponding known result for rows. 

Elementary Row Operations 

The next theorem shows how an elementary row operation on a matrix affects the value of its determinant. 


THEOREM 2.2.3 





Let Abe an nx.n Matrix. 

(a) IfB is the matrix that results when a single row or single column of A is multiplied by a scalar k, then det(5) = k det (A)- 


(b) IfB is the matrix that results when two rows or two columns of A are interchanged, then det(5) = — det (A)- 


(c) IfB is the matrix that results when a multiple of one row of A is added to another row or when a multiple of one column is 
added to another column, then det (B) = det(A)- 


We omit the proof but give the following example that illustrates the theorem for 3 x 3 determinants. 


EXAMPLE 1 Theorem 2.2.3 Applied to 3 x 3 Determinants 


We will verify the equation in the first row of Table 1 and leave the last two for the reader. By Theorem 2.1.1, the determinant of B 
may be found by cofactor expansion along the first row: 


det (5) = 


ka\\ 

s 

to 

ka\ 2 

<*21 

<*22 

<*22 

<*31 

<*22 

<*33 


= i<3llCn -h 12^12 ^33^13 

= k{a\\C\ \ -V anCn + a 33 Ci3) 


= k det (A) 


since C\2* anc ^ &13 not depend on the first row of the matrix, and A and B differ only in their first rows. 


Table 1 


Relationship Operation 


jtt* 11 

ka 12 

ka\ 2 


<*w 

<*12 

<*13 

<*21 

<*22 

<*23 

= k 

<*21 

<*22 

<*23 

<*31 

<*32 

<*33 


<* 3 \ 

<*32 

<*23 


det (B) = £det (A) 


<*21 

<*22 

<*23 


<*11 

<*12 

<*12 

<*11 

<*12 

<*13 

= — 

<*21 

<*22 

<*23 

<*31 

<*22 

<*32 


<*31 

<*32 

<*33 


det(5) = — det (A) 


<*11 1 ka 2 i 

<*12 - kkt * 22 

*13 + kaji 


an 

<*12 

<*13 

<*21 

<*22 

<*22 

= 

<*21 

<*22 

<*23 

<*31 

<*22 

<*33 


<*31 

<*32 

<*33 


The first row of A is multiplied by k- 


The first and second rows of A are interchanged. 


A multiple of the second row of A is added to the first row. 


det(5) = det(A) 


♦ 






















Rem As illustrated by the first equation in Table 1, part ( a ) of Theorem 2.2.3 enables us to bring a “common factor” from any 
row(or column) through the determinant sign. 

Elementary Matrices 

Recall that an elementary matrix results from performing a single elementary row operation on an identity matrix; thus, if we let 
A = l n in Theorem 2.2.3 [so that we have det(-d) = det(/ H ) = 1], then the matrix B is an elementary matrix, and the theorem yields the 
following result about determinants of elementary matrices. 


THEOREM 2.2.4 


Let E-be an nxn elementary matrix. 

( a ) If E results from multiplying a row of [ by b then det (E) = b 

(b) If E results from interchanging two rows of [ n , then det (5) = — 1- 

(c) If E results from adding a multiple of one row of[ n to another , then det(fi) = 1- 


EXAMPLE 2 Determinants of Elementary Matrices 


The following determinants of elementary matrices, which are evaluated by inspection, illustrate Theorem 2.2.4. 


1 

0 

0 

0 


0 

0 

0 

1 


1 

0 

0 

7 

0 

3 

0 

0 

= 3, 

0 

1 

0 

0 

= - 1, 

0 

1 

0 

0 

0 

0 

1 

0 

0 

0 

1 

0 

0 

0 

1 

0 

0 

0 

0 

1 


1 

0 

0 

0 


0 

0 

0 

1 


The second raw of/4 The first and last rows of 7 times the last row of/4 

was multiplied by 3 . /4 were interchanged. was added to the first row. 


♦ 


Matrices with Proportional Rows or Columns 

If a square matrix A has two proportional rows, then a row of zeros can be introduced by adding a suitable multiple of one of the rows 
to the other. Similarly for columns. But adding a multiple of one row or column to another does not change the determinant, so from 
Theorem 2.2.1, we must have det(j4) = 0- This proves the following theorem. 


THEOREM 2.2.5 


If A is a square matrix with two proportional rows or two proportional columns , then det (A) = 0- 


EXAMPLE 3 Introducing Zero Rows 














The following computation illustrates the introduction of a row of zeros when there are two proportional rows: 


1 

3 

-2 

4 


1 

3 

-2 

4 

2 

6 

-4 

8 


0 

0 

0 

0 

3 

9 

1 

5 


3 

9 

1 

5 

1 

1 

4 

8 


1 

1 

4 

8 


The second row is 2 times the 
first, so we added — 2 times 

the first row to the second to 
introduce a row of zeros. 


Each of the following matrices has two proportional rows or columns; thus, each has a determinant of zero. 


-1 4 

-2 8 



-2 7 
8 5 
-4 3 


3-1 4-5 

6-2 5 2 

5 8 14 

-9 3 -12 15 


Evaluating Determinants by Row Reduction 


♦ 


We shall now give a method for evaluating determinants that involves substantially less computation than the cofactor expansion 
method. The idea of the method is to reduce the given matrix to upper triangular form by elementary row operations, then compute the 
determinant of the upper triangular matrix (an easy computation), and then relate that determinant to that of the original matrix. Here 
is an example: 


EXAMPL Using Row Reduction to Evaluate a Determinant 


Evaluate det(2l) where 



1 5 
-6 9 
6 1 


Solution 

We will reduce A to row-echelon form (which is upper triangular) and apply Theorem 2.2.3: 


det(j4) = 


0 

1 

5 


3 

-6 

9 

3 

-6 

9 

= — 

0 

1 

5 

2 

6 

1 


2 

6 

1 


= -3 


= -3 


= -3 


-2 

1 

6 

-2 

1 


3 
5 

10 -5 


1 -2 

0 1 
0 


3 

1 5 

0 -55 


The first and second rows of 
A were interchanged. 

A common factor of 3 from 
the first row was taken 
through the determinant sign. 

— 2 times the first row was 
added to the third row. 

— 10 times the second row 
was added to the third row. 


= ( - 3)( - 55) 


1 -2 

0 1 

0 0 


A common factor of — 55 
from the last row was taken 
through the determinant sign. 


= ( — 3) ( — 55) (1) = 165 


Remar The method of row reduction is well suited for computer evaluation of determinants because it is computationally efficient 




























and easily programmed. However, cofactor expansion is often easier for hand computation. 


EXAMPLE 5 Using Column Operations to Evaluate a Determinant 


Compute the determinant of 


10 0 3 

2 7 0 6 

0 6 3 0 

7 3 1-5 


Solution 


This determinant could be computed as above by using elementary row operations to reduce A to row-echelon form, but we can put A 
in lower triangular form in one step by adding _ 3 times the first column to the fourth to obtain 


det(j4) = det 


1 

2 

0 

7 


0 

7 

6 

3 


0 

0 

3 

1 


0 

0 

0 

-26 


= (1)(7)(3)( — 26) = —546 


This example points out the utility of keeping an eye open for column operations that can shorten computations. 


♦ 


Cofactor expansion and row or column operations can sometimes be used in combination to provide an effective method for 
evaluating determinants. The following example illustrates this idea. 


Row Operations and Cofactor Expansion 


Evaluate det(j4) where 

'3 5 -2 6‘ 

j _ 1 2 — 1 1 

2 4 15 

3 7 5 3 

Solution 

By adding suitable multiples of the second row to the remaining rows, we obtain 












det(-d) = 


0 

1 

0 

0 


-1 


1 

3 

2 


1 

1 

0 


3 

3 

1 


8 

0 

- 1 

1 

3 


0 

3 

3 


1 

8 

0 


- 1 

1 

3 


0 

3 

3 


0 

9 

3 


(-1) 

3 

3 


9 

3 


18 





Cofactor expansion along the 
first column 

We added the first row to the 
third row. 

Cofactor expansion along the 
first column 


♦ 


Exercise Set 2.2 



Click here for Just Ask! 


Verify that det(A) = det(A T ) for 


(a) 


A = 


-2 3 
1 4 


(b) 2-13 

A= 1 2 4 

5-3 6 


Evaluate the following determinants by inspection. 

2 . 


(a) 


17 

5 



0 

o 

(b) 


0 i 


-8 

ft 1 


7 

0 - 


9 

5 

(c) 

-2 

1 3 


1 

-7 4 


-2 

1 3 


0 

0 

0 

1 





















(d) 1 -2 3 

2-4 6 
5-8 1 

Find the determinants of the following elementary matrices by inspection. 

3 . 

(a) [l 0 0 0 

0 1 0 0 

0 0-50 
0 0 0 1 


(b) 10 0 0 

0 0 10 
0 10 0 

0 0 0 1 

(c) r o o 

0 1 0 
0 0 1 
0 0 0 

In Exercises 4-11 evaluate the determinant of the given matrix by reducing the matrix to row-echelon form. 

3 6 — 9 

4- 0 0-2 

-2 1 5 

'0 3 r 

5 - 112 

3 2 4 

1-3 0' 

6.-2 4 1 

5-2 2 

'3-69 
7- - 2 7 - 2 

0 1 5 

[1-2 3 

8 * 5-9 6 

-1 2-6 
2 8 6 

[2 1 3 l' 

9- 1011 
0 2 10 
0 12 3 







10 . 


0 111 


1 I 

2 2 


1 1 


1 i « 


4 4 oo 


n. 


12 . 


1 

3 

1 

5 

3' 

-2 

-7 

0 

-4 

2 

0 

0 

1 

0 

1 

0 

0 

2 

1 

1 

0 

0 

0 

1 

1 



a b 

c 


Given that 

d e 

/ 

— _ 



g h 

i 


(a) 

d 

e f 




g 

h i 




a 

b c 



(b) 

3a 

3b 

3c 


— d — 

e 

-I 


4g 

4k 

4 i 

(c) 

ad g b 

+ k 

! C 


d 


s 

J 


g 


h 

i 

(d) 1 


3 a 

-3b 


= — 6, find 


d 

g-4d 


' / 

■ 4e ; — 4/ 


Use row reduction to show that 


13 . 


1 1 1 

a b c 

2,2 2 

a b c 


= (b — a)(c — a) (c — b) 


14 . 


Use an argument like that in the proof of Theorem 2.1.3 to show that 


(a) 


det 


0 0 a i3 

0 @22 a 23 

a 31 a 22 fl 33 


= — a \ 2 a 22 a 3 \ 







det 


= ai4fl23«32«41 


0 0 0 ^14 

0 0 «23 <324 

0 <332 <333 1334 

<341 a 42 fl 43 <*44 


Prove the following special cases of Theorem 2.2.3. 


(a) 

«21 

a 22 

<*23 


<311 

<312 

<313 


All 

<312 

<3 13 

= — 

<321 

«22 

a 23 


<331 

a 32 

<*33 


<331 

«32 

<333 


(b) 

<311 H *B2i 

<312 4- kct22 

<3i3 4- ka23 


<311 

<312 

<313 


«21 

<322 

<323 

= 

<321 

<322 

<323 


<331 

<332 

<333 


<331 

<332 

<333 


Repeat Exercises 4-7 using a combination of row reduction and cofactor expansion, as in Example 6. 

16 . 


Repeat Exercises 8-11 using a combination of row reduction and cofactor expansion, as in Example 6. 

17 . 


Discussion 

Discovery 


In each part, find det(j4) by inspection, and explain your reasoning. 

18 . 


(a) 001 

^=010 
1 0 0 


(b) 


A = 


0 0 
0 0 
0 1 
1 0 


0 1 
1 0 
0 0 
0 0 


19 . 


By inspection, solve the equation 


x 5 7 

0x4-1 6 


= 0 


0 0 2x -1 


Explain your reasoning. 


20 . 


(a) By inspection, find two solutions of the equation 






















(b) Is it possible that there are other solutions? Justify your answer. 


How many arithmetic operations are needed, in general, to find det(-d) by row reduction? By cofactor 
21* expansion? 
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In this section we shall develop some of the fundamental properties of the 
determinant function. Our work here will give us some further insight into the 
PROPERTIES OF TH E relationship between a square matrix and its determinant. One of the 
DETERMINANT immediate consequences of this material will be the determinant test for the 

FU NOTION invertibility of a matrix. 


Basic Properties of Determinants 

Suppose that A and B are n x n matrices and k is any scalar. We begin by considering possible relationships between det(44)> 
det (B), and 

det(£j4), det(44-|-5), and det(44£?) 

Since a common factor of any row of a matrix can be moved through the det sign, and since each of the n rows in fcA has a 
common factor of k, we obtain 


det(£j4) —k n det(-d) 

For example, 


ka ii 

ka 12 

r.'~\ 

i 1 


a n 

a \2 

a\3 

ka 2i 

ka22 

kajs 

C’"i 

II 

a 2\ 

a 22 

a 23 

k&2 1 


kaji 


a 31 

a 32 

a 33 


( 1 ) 


Unfortunately, no simple relationship exists among det(j4), det(i?)> and det(24 f B)- In particular, we emphasize that 
det(j4 -1- B ) will usually not be equal to det(j4) 4 det(5) • The following example illustrates this fact. 


det(A + B) * clet(A) + det (S) 


Consider 


1 

2 5 


3 

1 3 


A I 5 = 


4 

3 


3 

8 


We have det(A) = 1, det(5) = 8, and det(,4 VE) = 23; thus 

det (44 4 B) ^ det (44) -I- det (B) 

♦ 


In spite of the negative tone of the preceding example, there is one important relationship concerning sums of determinants 
that is often useful. To obtain it, consider two 2x2 matrices that differ only in the second row: 


AH <312 
<321 13 22 


and 


ail a 12 

*21 *22 


We have 



















Thus 


det(A) + det (B) = (a 11 ^ 22 -^ 12 ^ 21 ) + 011*22 - 12*21) 
= ^ll (^22 T * 22 ) -^ 12(^21 T * 2 l) 

_ J fl ll a 12 

fl 21 -I" *21 fl 22 "I- *22 


det 


flU flJ2 
<321 &22 


4 det 


ail 

^21 


a Y2 

= det 

an 

fl 12 

^22 

<* 2 i + *21 

3 22 1 &22 


This is a special case of the following general result. 


THEOREM 2.3.1 


Let A B» and CJ *c n x n matrices that differ only in a single row, say the yth, and assume that the rth row of CJ can be 
obtained by adding corresponding entries in the yth rows of A and £’■ Then 

det (C) = det(A) + det (5) 

The same result holds for columns. 


Using Theorem 2.3.1 


By evaluating the determinants, the reader can check that 



‘17 5 


'l 7 5" 


'l 7 5' 

det 

2 0 3 

= det 

2 0 3 

+ det 

2 0 3 


1 + 0 44 1 7 + (- 1) 


1 4 7 


0 1-1 


Determinant of a Matrix Product 

When one considers the complexity of the definitions of matrix multiplication and determinants, it would seem unlikely that 
any simple relationship should exist between them. This is what makes the elegant simplicity of the following result so 
surprising: We will show that if A and B are square matrices of the same size, then 

det(Ai?) = det(A) det(Z?) ^ 

The proof of this theorem is fairly intricate, so we will have to develop some preliminary results first. We begin with the 
special case of 2 in which A is an elementary matrix. Because this special case is only a prelude to 2, we call it a lemma. 


LEMMA 2.3.2 


If B is an nxn matrix andE is an n x n elementary matrix, then 

det (EB) = det (E) det (B) 




















Proof We shall consider three cases, each depending on the row operation that produces matrix E- 


Case 1. If E results from multiplying a row of / by fc, then by Theorem 1.5.1, EB results from B by multiplying a row by 
so from Theorem 2.2.3 a we have 


det(.£i?) = k det(5) 

But from Theorem 2.2.4a we have det(£) =k, so 

det (EB) = det(£)det(5) 

Cases 2 and 3. The proofs of the cases where E results from interchanging two rows of / or from adding a multiple of 
one row to another follow the same pattern as Case 1 and are left as exercises. 


Remark It follows by repeated applications of Lemma 2.3.2 that if B is an n x n matrix and E\> E r are n x n 

elementary matrices, then 


dti{E\E 2 "E r B) — det(i?i) det(52) det(iiV) det(£?) 


(3) 


For example, 


fei{E\E 2 B) — det(5i) det(i?2-£) = det(5'i)det(5'2)det(5) 


Determinant Test for Invertibility 

The next theorem provides an important criterion for invertibility in terms of determinants, and it will be used in proving 2. 


THEOREM 2.3.3 


A square matrix A is invertible if and only if dct(A) * 0- 


Proof Let R be the reduced row-echelon form of A- As a preliminary step, we will show that det (21) an d det(i?) are both 
zero or both nonzero: Let E\-> Ej-’ ■■■■> E r be the elementary matrices that correspond to the elementary row operations that 
produce R from A- Thus 


and from 3, 


R — E r "E2E\A 


det(iS) = det(£ r )‘''det(.£ 2 )det(i?i)det( 2 l) 

But from Theorem 2.2.4 the determinants of the elementary matrices are all nonzero. (Keep in 
mind that multiplying a row by zero is not an allowable elementary row operation, so k* 0 in this 
application of Theorem 2.2.4.) Thus, it follows from 4 that det(^) and det(i?) are both zero or both 
nonzero. Now to the main body of the proof. 

If A is invertible, then by Theorem 1.6.4 we have R = /, so det(i?) = 1^0 and consequently det(2l) ^ 0- Conversely, if 



det(A) 5 t 0, then det(3?) * 0, so R cannot have a row of zeros. It follows from Theorem 1.4.3 that R = /, so A is invertible by 
Theorem 1.6.4. 


It follows from Theorems Theorem 2.3.3 and Theorem 2.2.5 that a square matrix with two proportional rows or columns is 
not invertible. 


Determinant Test for Invertibility 


Since the first and third rows of 


A = 


1 

1 

2 


2 3 
0 1 
4 6 


are proportional, det (A) = 0- Thus A is not invertible. 


♦ 


We are now ready for the result concerning products of matrices. 


THEOREM 2.3.4 


If A and B are square matrices of the same size, then 

det(AS) = det (A) det(,S) 


Proof We divide the proof into two cases that depend on whether or not A is invertible. If the matrix A is not invertible, 
then by Theorem 1.6.5 neither is the product AB • Thus, from Theorem 2.3.3, we have det(A5) = 0 and det (A) = 0, so it 
follows that det(AS) = det(A) det(£?). 


Now assume that A is invertible. By Theorem 1.6.4, the matrix A is expressible as a product of elementary matrices, say 


A — E\E2“E r 


(5) 


so 


AB — E\E2"EyB 

det(AS) = detOSi) det(^)-det(5 f ) det(5) 


Applying 3 to this equation yields 
and applying 3 again yields 

det (AS) = &zt{E\E 2 "E r ) det (B) 
which, from 5, can be written as det (AS) = det (A) det (B). 


Verifying That det (AS) = det (A) det(S) 









Consider the matrices 


A = 


3 

2 


1 

1 


B = 


-1 3 
5 3 



17 

14 


We leave it for the reader to verify that 

det(j4) = 1, det(5) = - 23, and det(^) = - 23 
Thus det(AE?) = det(Al) det(5), as guaranteed by Theorem 2.3.4. 


♦ 


The following theorem gives a useful relationship between the determinant of an invertible matrix and the determinant of its 
inverse. 


THEOREM 2.3.5 


If A is invertible, then 


det(j4 -1 ) = 


1 

det(j4) 


Since A~ { A = b it follows that det(A4 1 j 4) = det(Z). Therefore, we must have det(Al ^det^d) = 1. Since det(Al) 0, 
the proof can be completed by dividing through by det(Al)- 


Linear Systems of the Form Ac — Ax 

Many applications of linear algebra are concerned with systems of n linear equations in n unknowns that are expressed in the 
form 


j4x — Ax 


( 6 ) 


where A is a scalar. Such systems are really homogeneous linear systems in disguise, since 6 can be rewritten as Ax — Alx = 0 
or, by inserting an identity matrix and factoring, as 


(A/ —j4)x = 0 


(7) 


Here is an example: 


Finding a/ — A 


The linear system 


can be written in matrix form as 


x\ 3x2 = Axj 
4xj + 2x2 — A*2 


'1 

3" 

'*f 

= A 

"*r 

_4 

2_ 

/ 2 _ 


/2_ 


which is of form 6 with 


















This system can be rewritten as 


A = 


1 3 
4 2 


and 


x = 


*1 

x 2 


or 


or 


which is of form 7 with 


■*r 


'1 3' 

■*r 


'O' 

x 2_ 


_4 2 _ 

x 2_ 


_o_ 


'1 o' 

■*r 


'1 3' 

"*f 


'o' 

_° 1_ 

x 2_ 


_4 2_ 

x 2_ 


_0_ 


'A- 1 

-3 ' 

■*r 


'O' 

_ -4 

> 

1 

[■■■_■' 

1_ 

*2 


_o_ 


Xl-A = 


A- 1 
-4 


-3 
A — 2 


♦ 


The primary problem of interest for linear systems of the form 7 is to determine those values of A for which the system has a 
nontrivial solution; such a value of A is called a characteristic value or an eigenvalue* of A- If A is an eigenvalue of A, th en 
the nontrivial solutions of 7 are called the eigenvectors of A corresponding to A- 


It follows from Theorem 2.3.3 that the system (A/ — A)x = 0 has a nontrivial solution if and only if 

det(A7 — A) = 0 

This is called the characteristic equation of A, the eigenvalues of A can be found by solving this equation for \. 


Eigenvalues and eigenvectors will be studied again in subsequent chapters, where we will discuss their geometric 
interpretation and develop their properties in more depth. 


Eigenvalues and Eigenvectors 


Find the eigenvalues and corresponding eigenvectors of the matrix A in Example 5. 


Solution 


The characteristic equation of A is 

det(Ai - A) 


A- 1 
-4 


-3 
A — 2 


= 0 


or A 2 — 3A — 10 = 0 


The factored form of this equation is (A +2)(A-5) = 0> so the eigenvalues of A are A = — 2 and A = 5- 


By definition, 


x = 


*1 

x 2 


is an eigenvector of A if and only if x is a nontrivial solution of (\[ — A) x = 0; that is, 


'A- 1 

-3 ' 

'^r 


'O' 

_ -4 

> 

1 

[-■■j 

1 _ 

x 2 


_ 0 _ 


( 9 ) 












































If x — 2, then 9 becomes 


'-3 -3" 

'*f 


'0" 

-1 

i 

1 

_1 

/2_ 


_0_ 


Solving this system yields (verify) x\= —t>X 2 = t* so the eigenvectors corresponding to A = — 2 are the nonzero solutions 
of the form 


■*r 


" - f 

*2 


£ 


Again from 9, the eigenvectors of A corresponding to A = 5 are the nontrivial solutions of 


i — 

m 

1 

^r 

i _ 

■*r 


'O' 

— i 

^r 

1 

_ i 

*2 


_o_ 


We leave it for the reader to solve this system and show that the eigenvectors of A corresponding to A = 5 are the nonzero 
solutions of the form 


x = 



♦ 


Summary 

In Theorem 1.6.4 we listed five results that are equivalent to the invertibility of a matrix A- We conclude this section by 
merging Theorem 2.3.3 with that list to produce the following theorem that relates all of the major topics we have studied 
thus far. 


THEOREM 2.3.6 

Equivalent Statements 

If A is an «x» matrix, then the following statements are equivalent. 

(a) A is invertible. 

(b) j4x = 0 has only the trivial solution. 

(c) The reduced row-echelon form of A is / . 

(d) A can be expressed as a product of elementary matrices. 

( e ) j4x = b is consistent for every n x 1 matrix £. 

(f) j4x = b has exactly one solution for every nxl matrix b. 

(g) det(j4) s* 0- 





















Exercise Set 2.3 



Click here for Just Ask! 


Verify that dzt(kA) — k n det(j4) for 

1 . 


(a) 


A = 


-1 2 

3 4 


; k = 2 


(b) 



-1 3 
2 1 
4 5 


■ k= -2 


Verify that det(AS) = det(A) det(B) for 



'2 1 O' 


'1 -1 3' 

A = 

3 4 0 

and B = 

7 1 2 


0 0 2 


5 0 1 


Is det(j4 I B) = det(j4) 4- det(£)? 

By inspection, explain why det(j4) = 0- 

'-2 8 1 4" 

A= 3 2 5 1 

1 10 6 5 

4-6 4-3 


Use Theorem 2.3.3 to determine which of the following matrices are invertible. 

4 . 


(a) 


1 

9 

8 


0 - 1 
-1 4 

9 -1 


(b) 


4 2 8 

-2 1-4 
3 1 6 


(c) 


ft 

l{2 


— \jl 0 

— 3\jl 0 
-9 0 




















Let 


-3 0 1 
5 0 6 
8 0 3 


5 . 


A = 


a b c 
d s f 


g h i 


Assuming that det(j4) = - 1, find 


(a) det(3j4) 

(b) det(j4 1 ) 

(c) det(2j4 -1 ) 

(d) det((2j4) _1 ) 

(e) a g d 

det b h e 

c i f 


6 . 


Without directly evaluating, show that x = 0 and x = 2 satisfy 


x 2 

1 1 

0 -5 


= 0 


7 . 


Without directly evaluating, show that 


det 


b H c c \- a b -\ a 

a b c 

1 1 1 


= 0 


In Exercises 8-11 prove the identity without evaluating the determinants. 


8 . 


ai b i 

a\ + &i 4 c i 


a x 

*1 

ci 


@2 &2 

a 2 4 i*2 1 


= 

&2 

b>2 

C2 


as b 3 

4 bs 1 



as 

h 



a\ 4 - b\ 

a\ ~b\ 

c 1 



ax 

hx 

ci 

a 2 4 - b'i 

a'l-bl 

c2 

= 

-2 

$2 

h 

c 2 

as 4 - bs 

as — bs 




as 

bs 

C3 


9 . 





10 . 


a\ 4 b\t <22 1 a 3 4- 




a\ 

®2 

a3 

a\t f&i ct 2 t 4- &2 a 3 M &3 

= (1- 

t 2 ) 

h 

b 2 

b 2 

ci C 2 c 3 




£1 

c 2 

£3 

ct 1 &i \ ta\ c\-\-rb\+sa\ 


a\ 

a 2 

a 3 



&2 b2 F ta2 C 2 4- rbi 4 sa' A 

= 

bx 

h 

h 



0,2 b 2 1 to,2 C 2 4 rb2 4 sct2 


£l 

£2 

£3 




For which value(s) of k does A fail to be invertible? 

12 . 


(a) 


A = 


k -3 
-2 


-2 
k -2 


(b) 12 4 

A= 3 16 
k 3 2 


Use Theorem 2.3.3 to show that 

2 .? -2 

sin a sin .i sin 7 

2 2j 2 

cos cr cos .? cos 7 

1 1 1 

is not invertible for any values of a, ft, and 7 . 


Express the following linear systems in the form (Al — A) x = 0. 

14. 


(a) 

(b) 

(c) 


xi 4 

2?: 2 = Xt 1 

2xi 4 

to 

II 

to 

2xi + 

3x2 = Axi 

4xi 4 

3x 2 = Ax 2 

3xi 4 

X 2 =Axi 

-5xi 

- 3x2 = Ax2 


For each of the systems in Exercise 14, find 

15. 

(i) the characteristic equation; 

(ii) the eigenvalues; 


(iii) the eigenvectors corresponding to each of the eigenvalues. 



Let A and B be hxm matrices. Show that if A is invertible, then det(5) = det(A ^ BA). 

16 . 


17. 

(a) Express 


a\Ab\ c\-\-d\ 

32 + ^2 c 2 4 d -2 

as a sum of four determinants whose entries contain no sums. 

(b) Express 


a\ -\- b\ 

Cl - 

1 d\ 

Cl 4/l 

<32 4- &2 

c 2 ~ 

■ d-2 

C2 + /2 

<33 -1- &3 

c 3 ~ 

1 <s?3 

C3 4/3 


as a sum of eight determinants whose entries contain no sums. 

Prove that a square matrix A is invertible if and only if A 7 A is invertible. 


Prove Cases 2 and 3 of Lemma Lemma 2.3.2. 

19. 


Discussion 

Discovery 


Let A and B be n x n matrices. You know from earlier work that AB and BA need not be equal. 

20. is the same true for det(AZ?) and det(ZM)? Explain your reasoning. 


Let A and B be n x n matrices. You know from earlier work that AB is invertible if A and B are 

21. invertible. What can you say about the invertibility of AB if one or both of the factors are 
singular? Explain your reasoning. 

Indicate whether the statement is always true or sometimes false. Justify each answer by giving 

22. a logical argument or a counterexample. 


(a) det(2A) = 2 det(A) 


(b) 




2 


(c) det(7 4 A) = U det(A) 


(d) If det(A) = 0, then the homogeneous system Ax = 0 has infinitely many solutions. 










Indicate whether the statement is always true or sometimes false. Justify your answer by giving 
23. a logical argument or a counterexample. 

(a) If det(A) = 0> then A is not expressible as a product of elementary matrices. 

(b) If the reduced row-echelon form of A has a row of zeros, then det(A) — 0- 


(c) The determinant of a matrix is unchanged if the columns are written in reverse order. 

(d) There is no square matrix A such that det(AA^) = — 1. 
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2.4 


A COMBINATORIAL 
APPROACH TO 
DETERMINANTS 


There is a combinatorial view of determinants that actually predates matrices. In 
this section we explore this connection. 


There is another way to approach determinants that complements the cofactor expansion approach. It is based on permutations. 



A permutation of the set of integers {1, 2,..., «} is an arrangement of these integers in some order without omissions or 
repetitions. 


Permutations of Three Integers 


There are six different permutations of the set of integers {1,2,3}. These are 


(1,2,3) (2,1,3) (3,1,2) 
(1,3,2) (2,3,1) (3,2,1) 


♦ 


One convenient method of systematically listing permutations is to use a permutation tree. This method is illustrated in our next 
example. 


Permutations of Four Integers 


List all permutations of the set of integers {1, 2, 3, 4}. 

Solution 

Consider Figure 2.4.1. The four dots labeled 1, 2, 3, 4 at the top of the figure represent the possible choices for the first number in 
the permutation. The three branches emanating from these dots represent the possible choices for the second position in the 
permutation. Thus, if the permutation begins (2, — , — , —), the three possibilities for the second position are 1, 3, and 4. The two 
branches emanating from each dot in the second position represent the possible choices for the third position. Thus, if the 
permutation begins (2, 3, — , — ), the two possible choices for the third position are 1 and 4. Finally, the single branch emanating 
from each dot in the third position represents the only possible choice for the fourth position. Thus, if the permutation begins with 
(2, 3, 4, — ), the only choice for the fourth position is 1. The different permutations can now be listed by tracing out all the 
possible paths through the “tree” from the first position to the last position. We obtain the following list by this process. 














(1,2, 3,4) 

(2, 1,3,4) 

(3, 1,2,4) 

(4, 

1,2,3) 

(1,2, 4, 3) 

(2, 1,4,3) 

(3, 1,4,2) 

(4, 

1,3,2) 

(1, 3, 2,4) 

(2, 3, 1,4) 

(3, 2 , 1,4) 

(4, 

2, 1, 3) 

(1, 3,4, 2) 

(2, 3,4, 1) 

(3, 2,4, 1 ) 

(4, 

2, 3, 1 ) 

(1,4, 2,3) 

(2,4, 1,3) 

(3,4, 1,2) 

(4, 

3, 1 , 2 ) 

(1,4, 3,2) 

(2,4, 3, 1) 

(3,4,2, 1) 

(4, 

3, 2 , 1 ) 


From this example we see that there are 24 permutations of {1, 2, 3, 4}. This result could have been anticipated without actually 
listing the permutations by arguing as follows. Since the first position can be filled in four ways and then the second position in 
three ways, there are 4 . 3 ways of filling the first two positions. Since the third position can then be filled in two ways, there are 
4 -3-2 ways of filling the first three positions. Finally, since the last position can then be filled in only one way, there are 
4 - 3 - 2 ■ 1 = 24 ways of filling all four positions. In general, the set { 1, 2,..., «} will have n(n — 1) (« — 2)—2 -1 = n \ different 
permutations. 

We will denote a general permutation of the set {1,2 ,...,«} by (j\, j2, Jn)- Here, jj is the first integer in the permutation, j 2 
is the second, and so on. An inversion is said to occur in a permutation ( Jh jf 2i j n ) whenever a larger integer precedes a 
smaller one. The total number of inversions occurring in a permutation can be obtained as follows: (1) find the number of integers 
that are less than j ^ and that follow in the permutation; (2) find the number of integers that are less than j 2 and that follow j 2 in 
the permutation. Continue this counting process for jj n _\- The sum of these numbers will be the total number of inversions 
in the permutation. 


Counting Inversions 


Determine the number of inversions in the following permutations: 


(a) ( 6 , 1, 3, 4, 5, 2) 


(b) (2, 4, 1, 3) 


(c) (1,2, 3, 4) 


Solution 


(a) The number of inversions is 5-1-04-1-1-1 + 1 


8 - 




























(b) The number of inversions is i _|_ 2 4 . 0 = 3- 


(c) There are zero inversions in this permutation. 


DEFINITION 


A permutation is called even if the total number of inversions is an even integer and is called odd if the total number of 
inversions is an odd integer. 


EXAMPLE 4 Classifying Permutations 


The following table classifies the various permutations of {1, 2, 3} as even or odd. 


Permutation 

Number of Inversions 

Classification 

(1,2, 3) 

0 

even 

(1,3,2) 

1 

odd 

(2, 1, 3) 

1 

odd 

(2, 3, 1) 

2 

even 

(3, 1, 2) 

2 

even 

(3, 2, 1) 

3 

odd 


Combinatorial Definition of the Determinant 

By an elementary product from an n x n matrix A we shall mean any product of n entries from A, no two of which come from the 
same row or the same column. 


EXAMPLE 5 Elementary Products 


List all elementary products from the matrices 


(a) 


AH <312 
«21 «22 















(b) 


3 11 

3 21 

3 31 


ai2 a 13 
3 22 13 23 
3 32 ajl 


Solution (a) 

Since each elementary product has two factors, and since each factor comes from a different row, an elementary product can be 
written in the form 

(31-(S'?- 

where the blanks designate column numbers. Since no two factors in the product come from the same column, the column numbers 
must be 12 or 21. Thus the only elementary products are an&22 and &\2 a 2\- 

Solution (b) 

Since each elementary product has three factors, each of which comes from a different row, an elementary product can be written 
in the form 

fli _(3'?_(3'?_ 

Since no two factors in the product come from the same column, the column numbers have no repetitions; consequently, they must 
form a permutation of the set {1, 2, 3}. These 3! = 6 permutations yield the following list of elementary products. 

3 11 3 22 3 33 3 12 3 21 3 33 3 13 3 21 3 32 
3 11 3 23 3 32 3 12 3 23 3 31 3 13 3 22 3 31 

♦ 

As this example points out, an n x n matrix A has n ! elementary products. They are the products of the form a ij where 

Ol> J 2- Jn) i s a permutation of the set ( 1, 2 , ...,«}• By a signed elementary product from A we shall mean an elementary 
product a\^a2j 2 "&nj„ multiplied by +1 or _ ]. We use the + if (j 1; j n ) is an even permutation and the if 

Oi. h. j») is an odd pennutation. 


Signed Elementary Products 


List all signed elementary products from the matrices 


'ail a 12' 
«21 13 22 


(b) 


an 

3 12 

3 13 

«21 

3 22 

3 23 

fl 31 

3 32 

3 33 


Solution 


Elementary Product Associated Permutation Even or Odd Signed Elementary Product 


( a ) 












Elementary Product 

Associated Permutation 

Even or Odd 

Signed Elementary Product 

a ll a 22 

(1,2) 

even 

^11^22 

a \2 a 2\ 

(2, 1) 

odd 

- fl 12 fl 21 


Elementary Product 

Associated Permutation 

Even or Odd 

Signed Elementary Product 

«11«22«33 

(1,2, 3) 

even 

«11^22^33 

£*11^23^32 

(1,3,2) 

odd 

-£*H£3!23^32 

fl 12 fl 21 fl 33 

(2, 1, 3) 

odd 

- fl 12 fl 21 fl 33 

£*12^23^31 

(2, 3, 1) 

even 

£3512^23^31 

a \2 a 2\ a 22 

(3, 1, 2) 

even 

£3S13£3S21£J!32 

fl 13 fl 22 fl 31 

(3, 2, 1) 

odd 

- fl 13 fl 22 fl 31 


We are now in a position to give the combinatorial definition of the determinant function. 


DEFINITION 


Let A be a square matrix. We define det(A) to be the sum of all signed elementary products from A- 


Determinants of 2 x 2 and 3x3 Matrices 


Referring to Example 6, we obtain 


(a) 


det 


a it a 12 
«21 ^22 


= a \\ a 22 — a \2 a 2\ 


(b) 

det 


fill a 12 fli3 
a 21 22 «23 

«3i fl32 £*33 


= ^11^22^33 +■ a \2 a 22 a 3\ H ^13 fl 21^32 


— fl 13 fl 22 fl 31 — fl 12 fl 21 fl 33 — fl ll fl 23 fl 32 


♦ 


Of course, this definition of det (A) agrees with the definition in Section 2.1, although we will not prove this. 





















These expressions suggest the mnemonic devices given in Figure 2.4.2. The formula in part (a) of Example 7 is obtained from 
Figure 2.4.2a by multiplying the entries on the rightward arrow and subtracting the product of the entries on the leftward arrow. 
The formula in part (b) of Example 7 is obtained by recopying the first and second columns as shown in Figure 2.4. 2b. The 
determinant is then computed by summing the products on the rightward arrows and subtracting the products on the leftward 
arrows. 





M'\ JHi 

X 


(a) Determinant of a 2X2 matrix \h) Determinant of a 3X3 matrix 

Figure 2.4.2 


We emphasize that the methods shown in Figure 2.4.2 do not work for determinants of 4 x 4 matrices or higher. 


Evaluating Determinants 


Evaluate the determinants of 


'3 r 


1 

2 

3' 

and B = 

-4 

7 

5 

-8 

6 

9 

4 -2 




Solution 



Using the method of Figure 2.4.2a gives 


det(j4) = (3) (— 2) — (1) (4) = — 10 
Using the method of Figure 2.4.2 b gives 

det(5) = (45) + (84) + (96) - (105) - ( - 48) - ( - 72) = 240 


♦ 


The determinant of A may be written as 

det(j4)=E±flij 1 fl 2 j 2 -fl«j B (i 

where E indicates that the terms are to be summed over all permutations (j j r ..., j n ) and the + or is selected in each term 
according to whether the permutation is even or odd. This notation is useful when the combinatorial definition of a determinant 
needs to be emphasized. 


Remark Evaluating determinants directly from this definition leads to computational difficulties. Indeed, evaluating a 4 x 4 














determinant directly would involve computing 41 = 24 signed elementary products, and a ] 0 x 10 determinant would require the 
computation of 10! = 3, 628, 800 signed elementary products. Even the fastest of digital computers cannot handle the computation 
of a 25 x 25 determinant by this method in a practical amount of time. 


Exercise Set 2.4 



Click here for Just Ask! 


Find the number of inversions in each of the following permutations of {1, 2, 3, 4, 5}. 

1 . 

(a) (4 1 3 5 2) 

(b) (5 342 1) 

(c) (3 254 1) 

(d) (543 2 1) 

(e) (1 2 3 4 5) 

(f) (1 423 5) 


Classify each of the permutations in Exercise 1 as even or odd. 

2 . 


In Exercises 3-12 evaluate the determinant using the method of this section. 


3. 


3 5 
-2 4 


4. 


4 1 
8 2 


5. 


-5 6 

-7 -2 


6 . 


4 


7 . 


a -3 5 

-3 a —2 














8 . 


-2 7 
5 1 
3 8 


6 

-2 

4 


9. 


-2 1 
3 5 
1 6 


4 

-7 

2 


10 . 


-1 1 2 

3 0-5 
1 7 2 


11 . 




12 . 


c 

2 

4 


-4 3 

1 c 2 
c — 1 2 


13. 


Find all values of \ for which det(j4) = 0> using the method of this section. 


(a) 


A — 2 1 

-5 A-I 4 


(b) 


A —4 0 0 

0 A 2 
0 3 A- 1 


Classify each permutation of {1, 2, 3, 4} as even or odd. 

14. 


15. 

(a) Use the results in Exercise 14 to construct a formula for the determinant of a 4 x 4 matrix. 


(b) Why do the mnemonics of Figure 2.4.2 fail for a 4 x 4 matrix? 


Use the formula obtained in Exercise 15 to evaluate 

4-992 
-2 5 6 4 

1 2-5-3 

1-2 0-2 


17 


Use the combinatorial definition of the determinant to evaluate 



(a) 


0 0 
0 0 
0 0 
0 2 
5 0 


0 

0 

- 1 
0 
0 


0 -3 
-4 0 
0 0 
0 0 
0 0 


(b) 


5 

0 

0 

0 

0 


0 0 0 
0 0 0 
0 3 0 
0 0 1 
-2 0 0 


0 

-4 

0 

0 

0 


18. 


Solve for x- 


X - 1 


3 1 — x 



0 -3 

x — 6 
3 x-5 


Show that the value of the determinant 

sin# cos# 0 

— cos# sin# 0 

sin# —cos# sin# + cos# 1 


does not depend on #, using the method of this section. 


20 . 


Prove that the matrices 



and 



/ 


commute if and only if 


b a — c 
e d-f 


= 0 


Discussion 

Discovery 


Explain why the determinant of an « x « matrix with integer entries must be an integer, using the 
21. method of this section. 


What can you say about the determinant of an n x n matrix all of whose entries are 1? Explain your 
22. reasoning, using the method of this section. 


23. 

(a) Explain why the determinant of an n x n matrix with a row of zeros must have a zero 
determinant, using the method of this section. 

(b) Explain why the determinant of an n x n matrix with a column of zeros must have a zero 
determinant. 




















Use Formula 1 to discover a formula for the determinant of an n x n diagonal matrix. Express the 

24. formula in words. 

Use Formula 1 to discover a formula for the determinant of an nxn upper triangular matrix. 

25. Express the formula in words. Do the same for a lower triangular matrix. 
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Chapter 2 


Supplementary Exercises 


1. 


Use Cramer's rule to solve for x f and y' in terms of x and y. 


x = Jx' 


¥ 


y = jx* + p' 


Use Cramer's rule to solve for x ! and y ! in terms of x and y. 
x = x* cos 9 — y* sin# 
y — x 1 sin.9 -I- y 1 cos 9 


By examining the determinant of the coefficient matrix, show that the following system has a nontrivial solution if and 
3* only if a = 8. 

x 4- y 4- 02 — 0 
x 4- y 4- ,3z — 0 
a x + ,.5>’ 4- z — 0 

Let A be a 3 x 3 matrix, each of whose entries is 1 or 0. What is the largest possible value for det(A)? 

4. 


5. 


(a) For the triangle in the accompanying figure, use trigonometry to show that 


b cos 7 + c cos 8 = a 
c cos a. + a cos o = b 
a cos j8 + b cos a. = c 

and then apply Cramer's rule to show that 


cos a = 


, 2.2 2 

b I c — a 

2 be 


(b) Use Cramer's rule to obtain similar formulas for cos 8 and cos 7 . 



C 


Figure Ex-5 


6 . 


Use determinants to show that for all real values of X the only solution of 





is x = 0, y = 0- 


x — 2y — Xx 
x -y = \y 


7. 


Prove: If A is invertible, then a dj (A) is invertible and 


[adj(j4) ] _1 


1 

det(jd) 


A = adj(jd _1 


) 


Prove: If A is an n x n matrix, then det [ adj (j 4) ] = [ det(y4) ]” 1 

8 . 


9. (For Readers Who Have Studied Calculus) Show that if f 1 (x), / 2 OO’ g\ (x)> and g 2 (x) are differentiable functions, 
and if 


w= 

f 1 OO / 2 CO 

. then dW = 

r -s 

1—»■ 

-Hb 

to 

2 

+ 

/ lO) f 2( x ) 


glCO g2 OO 

dx 

glO) S2 00 


g' iW g' 20 ) 


10 . 

(a) In the accompanying figure, the area of the triangle ABC can be expressed as 


area ABC = area ADEC A area CEFB — area ADFB 

Use this and the fact that the area of a trapezoid equals ^ the altitude times the sum of 
the parallel sides to show that 


area ABC = 


*1 71 

x 2 y 2 
*3 73 


1 

1 

1 


Note In the derivation of this formula, the vertices are labeled such that the triangle is 
traced counterclockwise proceeding from to (* 2 , 72 ) to (* 3 . 73 )- F° r a clockwise 

orientation, the determinant above yields the negative of the area. 


(b) Use the result in (a) to find the area of the triangle with vertices (3, 3), (4, 0), ( _ 2, — 1) • 



Prove: If the entries in each row of an n x n matrix A add up to zero, then the determinant of A is zero. 

11 . 

Hint Consider the product AX , where X is the n x 1 matrix, each of whose entries is one. 















Let A be an n x n matrix and B the matrix that results when the rows of A are written in reverse order (last row becomes 

12. the first, and so forth). How are det(j4) and det(5) related? 

Indicate how a~^ will be affected if 

13. 

(a) the ith and yth rows of A are interchanged. 

(b) the ith row of A is multiplied by a nonzero scalar, c . 

(c) c times the ith row of A is added to the yth row. 


14. 


Let A be an n x n matrix. Suppose that is obtained by adding the same number t to each entry in the ith row of A and 
that $2 is obtained by subtracting t from each entry in the ith row of A- Show that det(j4) = ^ [det(i?i) 4 det(i?2) ] • 


Let 


15. 


a n 

a 12 

a 13 

a 21 

a 22 

a 22 

$31 

a 32 

a 33 


(a) Express det(A/ - A) as a polynomial p (A) = A 3 4- b \ 2 I cA H d. 

(b) Express the coefficients b and d in terms of determinants and traces. 


Without directly evaluating the determinant, show that 


sin ct 

cos ct 

sinter 4- 6 ) 

sin 8 

cos 8 

sin(;7 1 <5) 

sin 7 

cos 7 

sin (7 -1- S) 


Use the fact that 21,375, 38,798, 34,162, 40,223, and 79,154 are all divisible by 19 to show that 

17. 

2 13 7 5 

3 8 7 9 8 

3 4 16 2 

4 0 2 2 3 
7 9 15 4 

is divisible by 19 without directly evaluating the determinant. 

Find the eigenvalues and corresponding eigenvectors for each of the following systems. 

18. 

*2 4 9x3 = Axi 
xi I 4 x 2 —7x3 —^2 
xi — 3x3= Ax 3 


(a) 



(b) X2 +*3 = A*i 

*1 -*3=A*2 

*1 4- 5^2 + 3^3 — A*3 
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Chapter 2 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Section 2.1 

Tl. (Determinants) Read your documentation on how to compute determinants, and then compute several determinants. 


T2. (Minors, Cofactors, and Adjoints) Technology utilities vary widely in their treatment of minors, cofactors, and adjoints. 

For example, some utilities have commands for computing minors but not cofactors, and some provide direct commands for 
finding adjoints, whereas others do not. Thus, depending on your utility, you may have to piece together commands or do 
some sign adjustment by hand to find cofactors and adjoints. Read your documentation, and then find the adjoint of the matrix 
A in Example 6. 

Use Cramer's rule to find a polynomial of degree 3 that passes through the points (0, 1), (1 ; _ l), (2, — 1), and (3, 7). Verify 
T3* your results by plotting the points and the curve on one graph. 

Section 2.2 

Tl. (Determinant of a Transpose) Confirm part ( b ) of Theorem 2.2.3 using some matrices of your choice. 


Section 2.3 

Tl. (Determinant of a Product) Confirm Theorem 2.3.4 for some matrices of your choice. 

T2. (Determinant of an Inverse) Confirm Theorem 2.3.5 for some matrices of your choice. 

T3. (Characteristic Equation) If you are working with a CAS, use it to find the characteristic equation of the matrix A in 
Example 6. Also, read your documentation on how to solve equations, and then solve the equation det(A/ — A) = 0 for the 
eigenvalues of A- 


Section 2.4 


Tl. 


(Determinant Formulas) If you are working with a CAS, use it to confirm the formulas in Example 7. Also, use it to obtc 



the formula requested in Exercise 15 of Section 2.4. 


T2. (Simplification) If you are working with a CAS, read the documentation on simplifying algebraic expressions, and then use 
the determinant and simplification commands in combination to show that 

bed 

“ = (« 2 I 4 2 f C 2 +i 2 ) 2 

dab 

c —b a 

Use the method of Exercise T2 to find a simple formula for the determinant 
T3 ‘ {a I b ) 2 c 2 c 2 

a 2 (i-hc ) 2 a 2 

b 2 b 2 (c + a ) 2 

Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 




3 

CHAPTER 


Vectors in 2-Space and 3-Space 


NTRODUCTlOh Many physical quantities, such as area, length, mass, and temperature, are completely described 
once the magnitude of the quantity is given. Such quantities are called, scalars. Other physical quantities are not completely 
determined until both a magnitude and a direction are specified. These quantities are called vectors. For example, wind 
movement is usually described by giving the speed and direction, say 20 mph northeast. The wind speed and wind direction 
form a vector called the wind velocity. Other examples of vectors are force and displacement. In this chapter our goal is to 
review some of the basic theory of vectors in two and three dimensions. 

Note. Readers already familiar with the contents of this chapter can go to Chapter 4 with no loss of continuity. 
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3.1 

INTRODUCTION TO 
VECTORS (GEOMETRIC) 


In this section, vectors in 2-space and 3-space will be introduced geometrically, 
arithmetic operations on vectors will be defined, and some basic properties of 
these arithmetic operations will be established. 


Geometric Vectors 


Vectors can be represented geometrically as directed line segments or arrows in 2-space or 3-space. The direction of the arrow 
specifies the direction of the vector, and the length of the arrow describes its magnitude. The tail of the arrow is called the initial 
point of the vector, and the tip of the arrow the terminal point. Symbolically, we shall denote vectors in lowercase boldface type 
(for instance, a, k, v, w, and x). When discussing vectors, we shall refer to numbers as scalars. For now, all our scalars will be 
real numbers and will be denoted in lowercase italic type (for instance, a , k, v, w, and x). 


If, as in Figure 3.1.1a, the initial point of a vector v is A and the terminal point is B, we write 

v = AB 

Vectors with the same length and same direction, such as those in Figure 3.1.1b, are called equivalent. Since we want a vector to 
be determined solely by its length and direction, equivalent vectors are regarded as equal even though they may be located in 
different positions. If v and w are equivalent, we write 

v = w 




lb) Equivalent vectors 

Figure 3.1.1 


DEFINITION 


If v and w are any two vectors, then the sum v + w is the vector determined as follows: Position the vector w so that its initial 
point coincides with the terminal point of v. The vector v -|- w is represented by the arrow from the initial point of v to the 
terminal point of w (Figure 3.1.2a). 









In Figure 3.1.2Z? we have constructed two sums, v -f w (color arrows) and w 4 . v (gray arrows). It is evident that 

v + w = w -|- v 

and that the sum coincides with the diagonal of the parallelogram determined by v and w when these vectors are positioned so 
that they have the same initial point. 


The vector of length zero is called the zero vector and is denoted by 0- We define 

0 + v = v + 0 = v 

for every vector v. Since there is no natural direction for the zero vector, we shall agree that it can be assigned any direction that 
is convenient for the problem being considered. If v is any nonzero vector, then _ v , the negative of v, is defined to be the vector 
that has the same magnitude as v but is oppositely directed (Figure 3.1.3). This vector has the property 

v+ (-v) =0 

(Why?) In addition, we define _ 0 = 0- Subtraction of vectors is defined as follows: 



Figure 3.1.3 

The negative of v has the same length as v but is oppositely directed. 


DEFINITION 


If v and w are any two vectors, then the difference of w from v is defined by 

v — w = v + ( — w) 

(Figure 3.1.4a). 












To obtain the difference v _ without constructing _ w , position v and w so that their initial points coincide; the vector from the 
terminal point of w to the terminal point of v is then the vector v — w (Figure 3.1 Ab). 


DEFINITION 


If v is a nonzero vector and k is a nonzero real number (scalar), then the product tv is defined to be the vector whose length is 
|£| times the length of v and whose direction is the same as that of v if t > 0 and opposite to that of v if t < 0- We define 
tv = 0 if k = 0 or v = 0- 


Figure 3.1.5 illustrates the relation between a vector v and the vectors ^v, ( _ ]) v , 2v> and ( _ 3) v . Note that the vector ( _ i) v 
has the same length as v but is oppositely directed. Thus ( _ l) v is just the negative of v; that is, 

(-l)v= -v 




Figure 3.1.5 


A vector of the form tv is called a scalar multiple of v. As evidenced by Figure 3.1.5, vectors that are scalar multiples of each 
other are parallel. Conversely, it can be shown that nonzero parallel vectors are scalar multiples of each other. We omit the proof. 

Vectors in Coordinate Systems 

Problems involving vectors can often be simplified by introducing a rectangular coordinate system. For the moment we shall 
restrict the discussion to vectors in 2-space (the plane). Let v be any vector in the plane, and assume, as in Figure 3.1.6, that v has 
been positioned so that its initial point is at the origin of a rectangular coordinate system. The coordinates (vj, V 2 ) of the terminal 
point of v are called the components of v, and we write 


V= (vi, V}) 










Figure 3.1.6 


vj and v 2 are the components of v. 


If equivalent vectors, v and w, are located so that their initial points fall at the origin, then it is obvious that their terminal points 
must coincide (since the vectors have the same length and direction); thus the vectors have the same components. Conversely, 
vectors with the same components are equivalent since they have the same length and the same direction. In summary, two 
vectors 


are equivalent if and only if 


v= (vi, v 2 ) 


and w=(wi,w 2 ) 


vi =wi and v 2 = w 2 


The operations of vector addition and multiplication by scalars are easy to carry out in terms of components. As illustrated in 
Figure 3.1.7, if 

v = (vi, v 2 ) and w= (wi,w 2 ) 


then 


v 4 w = (v i + w i, v 2 I w 2 ) 


( 1 ) 



*■ 


Figure 3.1.7 

If v = (vi, v 2 ) and k is any scalar, then by using a geometric argument involving similar triangles, it can be shown (Exercise 16) 
that 


tv — Ofcvi, jfcv 2 ) 


( 2 ) 


(Figure 3.1.8). Thus, for example, if v = (1, — 2) and w = (7, 6), then 

t + w=(1, -2)4- (7, 6 ) = (1+7, -2 f 6) = (8,4) 


and 


4v = 4(l, — 2) = (4(1), 4( — 2)) = (4, -8) 


Since, v — w = v + ( — l)w» it follows from Formulas 1 and 2 that 

V-W = (vi -W\, v 2 -w 2 ) 


(Verify.) 














Figure 3.1.8 


Vectors in 3-Space 

Just as vectors in the plane can be described by pairs of real numbers, vectors in 3-space can be described by triples of real 
numbers by introducing a rectangular coordinate system. To construct such a coordinate system, select a point O, called the 
origin, and choose three mutually perpendicular lines, called coordinate axes, passing through the origin. Label these axes x, y, 
and z, and select a positive direction for each coordinate axis as well as a unit of length for measuring distances (Figure 3.1.9a). 
Each pair of coordinate axes determines a plane called a coordinate plane. These are referred to as the xy-plane, the xz-plane, 
and the yz-plane. To each point P in 3-space we assign a triple of numbers (x, y, z), called the coordinates ofP, as follows: Pass 
three planes through P parallel to the coordinate planes, and denote the points of intersection of these planes with the three 
coordinate axes by X, Y, and Z (Figure 3.1.96). The coordinates of P are defined to be the signed lengths 

x = OX, y = OY, z = OZ 

In Figure 3.1.10a we have constructed the point whose coordinates are (4, 5, 6) and in Figure 3.1.106 the point whose coordinates 
are (_3, 2, _4). 




Figure 3.1.9 




Rectangular coordinate systems in 3-space fall into two categories, left-handed and right-handed. A right-handed system has the 
property that an ordinary screw pointed in the positive direction on the z-axis would be advanced if the positive x-axis were 
rotated 90°. toward the positive y-axis (Figure 3.1.11a); the system is left-handed if the screw would be retracted (Figure 
3.1.116). 


















(f/) liiglu-handed 



Figure 3.1.11 


Remark In this book we shall use only right-handed coordinate systems. 

If, as in Figure 3.1.12, a vector v in 3-space is positioned so its initial point is at the origin of a rectangular coordinate system, 
then the coordinates of the terminal point are called the components of v, and we write 

Y= (vi, v 2 , v 3 ) 



If y — (vi, vj, vV) and w = ( Wli w 3 ) are two vectors in 3-space, then arguments similar to those used for vectors in a plane 
can be used to establish the following results. 


v and w are equivalent if and only ifv\ = wp V2 = w>2, and v 3 = w 3 
v + w = (v 1 + w 1, V2 I W2, V 3 4 w 3 ) 
kv = (ivj, kv2, itv 3 )» where k is any scalar 


Vector Computations with Components 











If V = (1, - 3, 2) and w = (4, 2, 1), then 

T + w=(5, -1,3), 2v=(2, -6,4), - w= (-4, - 2 , - 1 ), 

V- w = V f C-w) = C-3, -5,1) 


♦ 


Application to Computer Color Models 


Blue Cyan 

to. 0.1) to, 1.1) 



Colors on computer monitors are commonly based on what is called the RGB color model. Colors in this system are created 
by adding together percentages of the primary colors red (R), green (G), and blue (B). One way to do this is to identify the 
primary colors with the vectors 


r = (1.0,0) (pure red), 

g= (0,1,0) (pure green), 

b = (0, 0, 1) (pure blue) 

in r 3 and to create all other colors by forming linear combinations of r, g, and b using coefficients 
between 0 and 1, inclusive; these coefficients represent the percentage of each pure color in the 
mix. The set of all such color vectors is called RGB space or the RGB color cube. Thus, each color 
vector c in this cube is expressible as a linear combination of the form 

c = cji’ + C2Z + C3b 

= ci(l,0, 0)+c 2 (0, 1,0)4 c 3 (0. 0,1) 

= (ci, c'i, ci) 

where 0<c, <l- As indicated in the figure, the corners of the cube represent the pure primary colors 
together with the colors, black, white, magenta, cyan, and yellow. The vectors along the diagonal 
running from black to white correspond to shades of gray. 


Sometimes a vector is positioned so that its initial point is not at the origin. If the vector p^p^ has initial point P\{x\,y\,z\) and 
terminal point ^2 ( JC 2 » 72. z 2 )> then 

P \?2 = (* 2 -^1.72 —y\,zi -zi) 

That is, the components of p^Pj are obtained by subtracting the coordinates of the initial point from the coordinates of the 
terminal point. This may be seen using Figure 3.1.13: 













The vector p^p 2 ' s ^ ie difference of vectors Qp^ and qp*^ , so 

P\P2 = OP2 — OP\ = (x 2 ,y 2 ,z 2 ) ~ (,xi,yi,zi) = (x 2 ~ * 1 ,72 ~y\,z 2 ~z\) 


Finding the Components of a Vector 


The components of the vector v _ p^p 2 i n ifi a l point p^ (2, — 1, 4) and terminal point P 2 (l, 5, — 8) are 

v = (7-2, 5 - ( - 1), (-8) -4) = (5, 6 , - 12) 

♦ 


In 2-space the vector with initial point Pi (xj, jy i) ar| d terminal point P 2 (x 2 , y 2 ) is 

P\P2 = (x 2 -xi,y 2 -yi) 


Translation of Axes 

The solutions to many problems can be simplified by translating the coordinate axes to obtain new axes parallel to the original 
ones. 

In Figure 3.1.14a we have translated the axes of an ^-coordinate system to obtain an x'V-coordinate system whose origin Of is 
at the point ( Xi y) = (k, !)■ A point P in 2-space now has both (x, y ) coordinates and (x ; , y ! ) coordinates. To see how the two are 
related, consider the vector o ! P (Figure 3.1.146). In the xy-system its initial point is at k, l ) and its terminal point is at (x), (y), so 
0 ! P = (x — k, y — !)■ In the x^-system its initial point is at (0, 0) and its terminal point is at, (x ! , y f ), so 0 ! P = {x*y*)- 
Therefore, 

T" 1 - _ jf- -yf - y _ ^ 

These formulas are called the translation equations. 











I 


t' v 


crl 


tut 


o 

(fl) 



[h) 

Figure 3.1.14 


Using the Translation Equations 


Suppose that an ^-coordinate system is translated to obtain an -coordinate system whose origin has ^-coordinates 
(*> i) = (4, 1). 


(a) Find the ^^' -coordinates of the point with the ^-coordinates P(2, 0 )- 

(b) Find the ^-coordinates of the point with A^y-coordinates Q( -\,j)- 

Solution (a) 

The translation equations are 

^ y * — y _ ] 

so the yy-coordinates of p(2, 0 ) are x* = 2 — 4 = — 2 and y ! = 0 — 1 = —1. 

Solution (b) 

The translation equations in (a) can be rewritten as 

x = x ! 4 4, y=y ! 4 1 

so the ^-coordinates of Q are x = —144 = 3 and 4 = 541 = 6 - 
In 3-space the translation equations are 

x ! = x — k, y =y-l, z ! = z — m 


♦ 










where ( k , /, m ) are the vyz-coordinates of the xyz- origin. 


Exercise Set 3.1 


O 


Click here for Just Ask! 


1 . 


Draw a right-handed coordinate system and locate the points whose coordinates are 


(a) (3,4,5) 


(b) (-3,4,5) 

(c) (3, -4,5) 

(d) (3, 4, _ 5 ) 


(e) ( — 3, -4,5) 

(f) (-3,4, -5) 


(g) (3, -4,-5) 

(h) ( - 3 , _ 4 , - 5 ) 

(i) (-3,0,0) 

(j) (3,0,3) 

(k) ( 0 , 0 , _ 3 ) 

(l) (0,3,0) 


Sketch the following vectors with the initial points located at the origin: 

2 . 

(a) vi = (3,6) 


(b) v 2 = ( —4, - 8 ) 




(c) v 3 = ( —4, -3) 


(d) v 4 = (5, -4) 

(e) VJ = (3,0) 

(f) v 6 = (0, -7) 

(g) v 7 = (3, 4, 5) 

(h) vg = (3, 3, 0) 

(i) v 9 = (0, 0, - 3) 

Find the components of the vector having initial point p 1 and terminal point p 2 - 

(a) P 1 (4,8), J P 2 (3,7) 

(b) P { (3, -5),P 2 {-4, -7) 

(c) P 1 (-5,0),P 2 (-3, 1) 

(d) ^(0, 0 ),P 2 (a,b) 

(e) P { (3, - 1 , 2), (-2, 5, -4) 

(f) Pit-1,0, 2),P 2 (0, -1,0) 

(g) Py(a,b,c),P 2 (0, 0, 0) 

(h) Pi(0, 0, 0),P 2 (a,b,c) 

Find a nonzero vector u with initial point P( — 1,3, —5) such that 


(a) u has the same direction as v = (6, 7, — 3) 




(b) u is oppositely directed to y = (6 i 7 > _ 3) 


Find a nonzero vector u with terminal point Q(3, 0, — 5) such that 


(a) u has the same direction as v = ( 4 , — 2 , — 1 ) 


(b) u is oppositely directed to v = ( 4 , — 2 , — 1 ) 


Let u = ( — 3, 1, 2). v = (4, 0, - 8 ).and w =( 6 , -1, - 4 )-Find the components of 

6 . 


(a) v-w 

(b) 6 u + 2 v 

(c) - v + u 


(d) 5 (v — 4u) 


(e) — 3 (v — 8 w) 

(f) ( 2 u — 7w) — ( 8 v + u) 


Let u, v, and w be the vectors in Exercise 6 . Find the components of the vector x that satisfies 2u — v + x = 7x + 

7. 


Let u, v, and w be the vectors in Exercise 6 . Find scalars c\,c 2 , and c 3 such that 

ciu h C 2 V + C 3 W= (2, 0, 4) 


Show that there do not exist scalars c\, C 2 , and c 3 such that 
ci( - 2, 9, 6 ) +c 2 ( - 3, 2, 1) +c 3 (l, 7, 5) = (0, 5, 4) 


Find all scalars c\,C 2 , and c 3 such that 
10 * ci(l, 2, 0) + c 2 (2, L 1) + c 3 (0, 3, 1) = (0, 0, 0) 


Let P be the point (2, 3, _ 2) and Q the point (7, _ 4 , 1). 

11 . 


(a) Find the midpoint of the line segment connecting P and Q. 


(b) Find the point on the line segment connecting P and Q that is j of the way from P to Q. 


12 . 


Suppose an ^-coordinate system is translated to obtain an -coordinate system whose origin 0 ! has ^-coordinates (2, 
-3)- 


f f 

(a) Find the x y' -coordinates of the point P whose ^-coordinates are (7, 5). 


(b) Find the ^-coordinates of the point Q whose -coordinates are (_ 3, 6). 


(c) Draw the xy and x^-coordinate axes and locate the points P and Q. 

(d) If v = (3, 7) is a vector in the xy-coordinate system, what are the components of v in the x y'-coordinate system? 

(e) If v = (vi, Vj) is a vector in the xy-coordinate system, what are the components of v in the x r y-coordinate system? 


Let P be the point (1,3, 7). If the point (4, 0, _ 6) is the midpoint of the line segment connecting P and Q, what is Q! 

13. 

Suppose that an xyz -coordinate system is translated to obtain an x ! y ! z ! -coordinate system. Let v be a vector whose 
components are v = (v i , V 2 , vj) in the xyz-system. Show that v has the same components in the x y V-system. 

Find the components of u, v, u _|_ v , and u _ v for the vectors shown in the accompanying figure. 

15. 



Figure Ex-15 


Prove geometrically that if v = (v i, V 2 ), then kv = (kv\, kv-i)- (Restrict the proof to the case k > 0 illustrated in Figure 3.1.8. 
16* The complete proof would involve various cases that depend on the sign of k and the quadrant in which the vector falls.) 


Discussion 

Discovery 


Consider Figure 3.1.13. Discuss a geometric interpretation of the vector 

u=o>i + i(d>2-o>i) 


17 . 






Draw a picture that shows four nonzero vectors whose sum is zero. 

18 . 

If you were given four nonzero vectors, how would you construct geometrically a fifth vector that 

19. is equal to the sum of the first four? Draw a picture to illustrate your method. 

Consider a clock with vectors drawn from the center to each hour as shown in the accompanying 

20. figure. 


(a) What is the sum of the 12 vectors that result if the vector terminating at 12 is doubled in 
length and the other vectors are left alone? 

(b) What is the sum of the 12 vectors that result if the vectors terminating at 3 and 9 are each 
tripled and the others are left alone? 

(c) What is the sum of the 9 vectors that remain if the vectors terminating at 5, 11, and 8 are 
removed? 



Indicate whether the statement is true (T) or false (F). Justify your answer. 

21 . 

(a) If x + y = x 4- z, then y = x. 

(b) If n _|_ v — 0 , then an 4. hv = 0 for all a and b. 

(c) Parallel vectors with the same length are equal. 

(d) If ax — 0, then either a — 0 or x = 0- 


(e) If tm 4- bv = 0, then u and v are parallel vectors. 


(f) 


The vectors u = {\[2, \j?>) and v = 



are equivalent. 
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3.2 

NORM OF A VECTOR; 
VECTOR ARITHMETIC 


In this section we shall establish the basic rules of vector arithmetic. 


Properties of Vector Operations 

The following theorem lists the most important properties of vectors in 2-space and 3-space. 


THEOREM 3.2.1 

Properties of Vector Arithmetic 

Ifu, v, and w are vectors in 2- or 3-space and k and l are scalars, then the following relationships hold. 

(a) u T v = v I u 

(b) (u h v) + w = uT (v-I- w) 

(c) ii -l- 0 = 0 -l- n = ii 

(d) ii + ( — ii) = 0 

(e) Ar(M) = (kP)\\ 

(f) £(u -I- v) = jtii -I- ^rv r 

(g) (k I i)u = £u |- iu 

(h) hi = u 


Before discussing the proof, we note that we have developed two approaches to vectors: geometric , in which vectors are 
represented by arrows or directed line segments, and analytic, in which vectors are represented by pairs or triples of numbers 
called components. As a consequence, the equations in Theorem 1 can be proved either geometrically or analytically. To 
illustrate, we shall prove part ( b ) both ways. The remaining proofs are left as exercises. 


We shall give the proof for vectors in 3-space; the proof for 2 -space is similar. If u = (u \, U 2 , uf) 





V= (vi, V2, V 3 ), and w = ( Wl> W2 , W3), then 


(u + v) 4w = [(ui,U 2,U3) + (v 1 .v 2 .v 3 )] 4- (w 1 .w 2 .w 3 ) 

= (ui f V 1 .U 2 + V2.U3. +V 3 ) +(w 1 .w 2 .w 3 ) 

= ([“1 + v i] +wi, [U2 4 v 2 ] 4 W2, [U3 4 v 3 ] 4 w 3 ) 
= (ui f [vi 4 wi], U 2 4 [v 2 4 w 2 ], u 3 4 [v 3 4 w 3 ]) 
= (ui,U 2 , U3) 4 (vi h wi, V2 4 W 2 , V3 4 W3) 

= u d (v 4 w) 


Proof of part (b) (geometric) 

Also, 

Therefore, 


Let u, v, and w be represented by PQ, QR, and rj? as shown in Figure 3.2.1. Then 

and u 4 (v 4 w) = PS 

and ( 114 v) -I- w = PS 


u 4 (v 4 w) = (u 4 v) 4 w 
Q 



v 4 w = QS 

u \- V =PR 


Figure 3.2.1 

The vectors u 4 (v 4 w) and ( u 4. v ) f w are equal. 


Remark In light of part (b) of this theorem, the symbol u 4 v 4 w is unambiguous since the same sum is obtained no matter 
where parentheses are inserted. Moreover, if the vectors u, v, and w are placed “tip to tail,” then the sum u 4 v I- w is the vector 
from the initial point of u to the terminal point of w (Figure 3.2.1). 

Norm of a Vector 

The length of a vector u is often called the norm of u and is denoted by ||u||. It follows from the Theorem of Pythagoras that the 
norm of a vector u = (ui, 113 ) i n 2 -space is 

l|u|| = 4 u 2 2 (1) 


(Figure 3.2.2 a). Let u = ( Uli U 3 ) be a vector in 3-space. Using Figure 3.2.2 b and two applications of the Theorem of 
Pythagoras, we obtain 

||u|| 2 = (OR) 2 + (RP) 2 = (OQ) 2 4 (OS ) 2 4 (RP) 2 = u\ 4u| 4u| 


Thus 


u 





2 

3 


( 2 ) 




A vector of norm 1 is called a unit vector. 




Global Positioning 

GPS () is the system used by the military, ships, airplane pilots, surveyors, utility companies, automobiles, and hikers to 
locate current positions by communicating with a system of satellites. The system, which is operated by the U.S. Department 
of Defense, nominally uses 24 satellites that orbit the Earth every 12 hours at a height of about 11,000 miles. These satellites 
move in six orbital planes that have been chosen to make between five and eight satellites visible from any point on Earth. 



To explain how the system works, assume that the Earth is a sphere, and suppose that there is an xyz-coordinate system with 
its origin at the Earth's center and its z-axis through the North Pole. Let us assume that relative to this coordinate system a 
ship is at an unknown point (. x , y, z) at some time t. For simplicity, assume that distances are measured in units equal to the 
Earth's radius, so that the coordinates of the ship always satisfy the equation 


+ v 



The GPS identifies the ship's coordinates ( x , y, z) at a time t using a triangulation system and computed distances from four 
satellites. These distances are computed using the speed of light (approximately 0.469 Earth radii per hundredth of a second) 
and the time it takes for the signal to travel from the satellite to the ship. For example, if the ship receives the signal at time t 
and the satellite indicates that it transmitted the signal at time ^q, then the distance d traveled by the signal will be 











d = 0.469(£ —^o) 

In theory, knowing three ship-to-satellite distances would suffice to determine the three unknown coordinates of the ship. 
However, the problem is that the ships (or other GPS users) do not generally have clocks that can compute t with sufficient 
accuracy for global positioning. Thus, the variable t must be regarded as a fourth unknown, and hence the need for the 
distance to a fourth satellite. Suppose that in addition to transmitting the time each satellite also transmits its coordinates ( 
jcq> yn, zg) at that time, thereby allowing d to be computed as 

d = /(*-* o ) 2 + 0 -^ o ) 2 + ( z -^ o ) 2 

If we now equate the squares of d from both equations and round off to three decimal places, then we obtain the 
second-degree equation 

(x-x n ) 2 + (y — .yn) 2 + (z-z n ) 2 = 0.22(t -tn) 2 

Since there are four different satellites, and we can get an equation like this for each one, we can produce four equations in 
the unknowns x, y, z, and Although these are second-degree equations, it is possible to use these equations and some 
algebra to produce a system of linear equations that can be solved for the unknowns. 


If (xi, y i, zi) and P-dj^i, y?, z?) are two points in 3-space, then the distance d between them is the norm of the vector p^p 2 
(Figure 3.2.3). Since 

P\P2 = (x2-x\,y2-y\,Z2-z\) 

it follows from 2 that 

d = /(*2-*l) 2 4 02—7l) 2 + (^2—zi) 2 (3) 

Similarly, if P\ (*1, y 1) and ^ 2 (^ 2 - P2) are P°i nts i n 2-space, then the distance between them is given by 

d = ^(*2—*l) 2 4 02-7l) 2 (4) 



Figure 3.2.3 

The distance between p 1 and i s the norm of the vector p^p^ 


Finding Norm and Distance 


The norm of the vector u=( — 3,2, 1) is 

ll«ll = /( —3) 2 -| (2) 2 -f (1) 2 = 

The distance d between the points P\{2, — 1, — 5) and P2{4, — 3, 1) is 

<3? =l /(4_2) 2 -|-( — 3-1- 1) 2 + (l + 5) 2 = ^44 = 2/11 

♦ 
















From the definition of the product £u, the length of the vector £u is |£| times the length of u. Expressed as an equation, this 
statement says that 


ll*n|l = |*|||u|| 

This useful formula is applicable in both 2-space and 3-space. 


Exercise Set 3.2 



Click here for Just Ask! 


Find the norm of v. 

1 . 


(a) v = (4, -3) 


(b) v = (2, 3) 


(c) v=( —5,0) 


(d) v = (2, 2, 2) 


(e) v=( —7,2, -1) 


(f) v= (0,6,0) 


Find the distance between and p 2 - 


(a) P 1 (3.4),P 2 (5.7) 


(b) p l (-3,6),P 1 (-\, -4) 


(c) Pj(7, -5, l).P 2 (-7, -2,-1) 

(d) P\(3, 3, 3). P 2 (6, 0, 3) 


3 . 


Let u = (2, — 2, 3). v = (1, — 3, 4),w = (3, 6, — 4)- I n eac h part, evaluate the expression. 




(a) 11“ I- V H 


(b) IN + IMI 

(c) || — 2u|| 4- 2||u|| 

(d) ||3u —5v h w|| 


If 11v11 = 2 and ||w|| = 3, what are the largest and smallest values possible for ||v — w||? Give a geometric explanation of your 
4. results. 

Let u = (2, 0, 4) and v = (1, 3, — 6)- In each of the following, determine, if possible, scalars k, l such that 

(a) ku + lv = (5, 9, -14) 

(b) foi + tv=(9, 15, -21) 

Let u = (2, 6, - 7), v =( - 1, - 1, 8), and * = 3- If (2, 14, 11) = jfcu + hr, what is the value of 17 

6 . 

Let v = ( — 1, 2, 5)- Find all scalars k such that ||£v|| = 4. 

7. 

Let u = (7 i — 3 r 1), v = (9, 6, 6), w = (2, 1, — 8). k — — 2, and / = j. Verify that these vectors and scalars satisfy the stated 
equalities from Theorem 1. 

(a) part ( b) 

(b) part ( e ) 

(c) part (j) 


(d) part (g) 




9 . 


( a ) Show that if v is any nonzero vector, then .. -.. v is a unit vector. 

IMI 

(b) Use the result in part (a) to find a unit vector that has the same direction as the vector v = (3, 4). 

(c) Use the result in part (a) to find a unit vector that is oppositely directed to the vector y = ( — 2, 3, — 6)- 


10 . 


(a) Show that the components of the vector v = (vj, V 2 ) in Figure Ex-10a are Vl _ ||vllcos 9 an d vj = ||v||sin0- 

(b) Let u and v be the vectors in Figure Ex-10/?. Use the result in part (a) to find the components of 4 U _ 5 V . 



j 

L-V 


S 

N is 

) r T 

/ / 

4/f 

fz 13 

/ 

1^* 


11 . 


12 . 


13 . 


14 . 


(/>) 

Figure Ex-10 

Let p 0 = ( lQi _y 0i zq) and p = ( x , y, z)- Describe the set of all points (x, y, z) for which ||p — poll = 1- 

Prove geometrically that if u and v are vectors in 2- or 3-space, then ||u -|- v|| < ||u|| 4- ||v||. 

Prove parts (a), (c), and ( e ) of Theorem 1 analytically. 

Prove parts ( d ), (g), and (h) of Theorem 1 analytically. 


Discussion 

Discovery 


For the inequality stated in Exercise 9, is it possible to have ||u 4 v|| = ||u|| 4- ||v||? Explain your 
15 . reasoning. 


16 . 

(a) What relationship must hold for the point p = (a, b, c ) to be equidistant from the origin 
and the *y-plane? Make sure that the relationship you state is valid for positive and 









negative values of a, b, and c. 


(b) What relationship must hold for the point p = (a, b, c ) to be farther from the origin than 
from the v Z -plane? Make sure that the relationship you state is valid for positive and 
negative values of a , b, and c. 


17 . 


(a) What does the inequality || x|| < 1 tell you about the location of the point x in the plane? 


(b) Write down an inequality that describes the set of points that lie outside the circle of 
radius 1, centered at the point xq. 


The triangles in the accompanying figure should suggest a geometric proof of Theorem 3.2.1 if) 
18 . for the case where k > 0 Give the proof. 



(a) 

Figure Ex-18 
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3.3 

DOT PRODUCT; 
PROJECTIONS 


In this section we shall discuss an important way of multiplying vectors in 
2-space or 3-space. We shall then give some applications of this 
multiplication to geometry. 


Dot Product of Vectors 

Let u and v be two nonzero vectors in 2-space or 3-space, and assume these vectors have been positioned so that their initial 
points coincide. By the angle between u and v, we shall mean the angle 0 determined by u and v that satisfies 0 < 9 < tt (Figure 
3.3.1). 




Figure 3.3.1 

The angle 0 between u and v satisfies 0 < Q < -a. 


DEFINITION 


If u and v are vectors in 2-space or 3-space and 0 is the angle between u and v, then the dot product or Euclidean inner 
product u. y is defined by 

_ f ll u ll ll v ll cos 0 if u 5* 0 and v 0 
10 if u = 0 or v = 0 


Dot Product 


As shown in Figure 3.3.2, the angle between the vectors u = (0, 0, 1) and v = (0, 2, 2) is 45°. Thus 

U V = Hull ||t||cos 9 = (|/o 2 + 0 2 +l 2 ) ( l/o 2 H 2 2 -I 2 2 ) J = 2 

















♦ 


Component Form of the Dot Product 

For purposes of computation, it is desirable to have a formula that expresses the dot product of two vectors in terms of the 
components of the vectors. We will derive such a formula for vectors in 3-space; the derivation for vectors in 2-space is 
similar. 

Let u = (u i, til, and v = (vj, V 2 , V 3 ) be two nonzero vectors. If, as shown in Figure 3.3.3, 0 is the angle between u and v, 

then the law of cosines yields 

ll-FSI | 2 = ll u l | 2 + 11 v 11 2 — 211 xi 111 | v|| cos 0 ( 2 ) 


Since pQ = v — u, we can rewrite 2 as 

||u||||v|| cos 0 = i(||u|| 2 4 - ||v|| 2 - ||v-u|| 2 ) 
or 

u-v=IC||u|| 2 -H|v|| 2 -||v-u|| 2 ) 

Substituting 

||u|| II v|| — VJ+V 2 +V 3 . 

and 

|| v “ u || 2 — (v 1 “ u 1) 2 + (v 2 -U 2 ) 2 + CV 3 -W 3) 2 

we obtain, after simplifying, 


u-v = uivi f ^2 v 2 + U3V3 


(3) 


Although we derived this formula under the assumption that u and v are nonzero, the formula is also valid if u = 0 ° r v = 0 
(verify). 



If 11 = (m, U 2 ) ar| d v = (vi, V 2 ) are two vectors in 2-space, then the formula corresponding to 3 is 






u ■ v = u\v\ f ^2V2 


Finding the Angle Between Vectors 

If u and v are nonzero vectors, then Formula 1 can be written as 


( 4 ) 



(5) 


EXAMPLE 2 Dot Product Using (3) 

Consider the vectors u = (2, —1,1) and v = (1, 1, 2)- Find u . v and determine the angle 0 between u and v. 

Solution 


u- v = uivi F U 2 V 2 + U 3 V 3 — (2) (1) + ( - 1) (1) 4- (1) (2) — 3 


For the given vectors we have ||u|| = ||v|| = ^ 6 , so from (5), 



Thus, 0 = 60 c - 


♦ 


EXAMPLE 3 A Geometric Problem 

Find the angle between a diagonal of a cube and one of its edges. 

Solution 

Let k be the length of an edge and introduce a coordinate system as shown in Figure 3.3.4. If we let = (k, 0, 0)> 
u 2 = ( 0 , k, 0 ), and U3 = ( 0 , 0 , k), then the vector 

(l = (At, k, k) = ui + u 2 + u 3 

is a diagonal of the cube. The angle 0 between d and the edge 11 j satisfies 



(mfs 1 ?) 1/3 


Thus 



54.74° 


Note that this is independent of k, as expected. 













iJt.fc.At 


(0,0. fr l 



The following theorem shows how the dot product can be used to obtain information about the angle between two vectors; it 
also establishes an important relationship between the norm and the dot product. 


THEOREM 3.3.1 


Let u and v be vectors in 2- or 3-space. 



(a) v - v = ||v|| 2 ; that is, ||v|| = (v - v) 



(b) If the vectors u and v are nonzero and 9 

is the angle between them, then 


8 is acute 

if and only if u - v > 0 


<9 is obtuse 

if and only if u ■ v < 0 


<35 

II 

if and only if u - v = 0 


Proof (a) Since the angle 0 between v and v is 0, we have 

v v = ||v||||v|| cos 8 — ||v|| 2 cosO = ||v|| 2 


Proof (b) Since 0 satisfies 0 < 8 < x, it follows that 0 is acute if and only if CO s 8 > 0, that 0 is obtuse if and only if CO s 8 < 0, 
and that Q = ^ j 2 if and only if C os 8 = 0- But cos 8 has the same sign as u . v since u - v = ||u|| ||v|| cos 8, ||u|| > 0, and ||v|| > 0. 
Thus, the result follows. 


Finding Dot Products from Components 


If u = (1, - 2, 3), T = C - 3, 4, 2), and w = (3, 6, 3), then 









u-v = (1)( — 3) + ( — 2) (4) 4- (3) (2) = — 5 
v-w=(-3)(3)-| (4)(6) + (2)(3) = 21 
u-w = (1)(3) -h ( — 2)(6) -h (3)(3) = 0 

Therefore, u and v make an obtuse angle, v and w make an acute angle, and u and w are perpendicular. 


♦ 


Orthogonal Vectors 

Perpendicular vectors are also called orthogonal vectors. In light of Theorem 3.3.1 b, two nonzero vectors are orthogonal if 
and only if their dot product is zero. If we agree to consider u and v to be perpendicular when either or both of these vectors is 
0, then we can state without exception that two vectors u and v are orthogonal ( perpendicular ) if and only if u - v = 0- To 
indicate that u and v are orthogonal vectors, we write u j_ v . 


A Vector Perpendicular to a Line 


Show that in 2-space the nonzero vector n — ( a , b) is perpendicular to the line ax -4 by f c = 0- 


Solution 

Let Pi (jq, _yi) and p 2 (x 2 , yf) distinct points on the line, so that 

ax 1 -I- by 1 + c = 0 
ax 2 4- by 2 + c = 0 

Since the vector p^p 2 = (x? — jq y 2 — >'i) runs a ^ on S the line (Figure 3.3.5), we need only show that n and p^p 2 are 
perpendicular. But on subtracting the equations in (6), we obtain 

tf(*2-*i) + b(y2-y\) = 0 

which can be expressed in the form 

(a,b)-(x2-x\,y2-y\) = § ^ n-P\P2 = § 

Thus n and p^p 2 are perpendicular. 

A >' 



♦ 

The following theorem lists the most important properties of the dot product. They are useful in calculations involving vectors. 


THEOREM 3.3.2 


Properties of the Dot Product 












Ifu, v, and w are vectors in 2-or 3-space and k is a scalar, then 

(a) u - v = v - u 

(b) u - (v -| w) = u - v -|- u - w 

(c) A;(u - v) = (Aii) - v = u - (Arv r ) 

(d) v - v > 0 if v * 0, and v - v = 0 if v = 0 


Proof We shall prove (c) for vectors in 3-space and leave the remaining proofs as exercises. Let u = (ui, uj, uf) and 
v = (vi, vO; t ^ ien 


Ar(u- v) = A:(uivi + U2V2 4 - W3V3) 

= (A^i)vi -I- {kit2)v2 -3- (kui)V2 
— (A11) - v 

Similarly, 

A;(u - v) — u - (Arv) 


An Orthogonal Projection 


In many applications it is of interest to “decompose” a vector u into a sum of two terms, one parallel to a specified nonzero 
vector a and the other perpendicular to a. If u and a are positioned so that their initial points coincide at a point Q, we can 
decompose the vector u as follows (Figure 3.3.6): Drop a perpendicular from the tip of u to the line through a, and construct 
the vector wj from Q to the foot of this perpendicular. Next form the difference 

w? = 11 — wi 

As indicated in Figure 3.3.6, the vector wj is parallel to a, the vector W 2 is perpendicular to a, and 

wi -I- W2 = wi + (u — wi) — u 

The vector wi is called the orthogonal projection of u on a or sometimes the vector component of u along a. It is denoted by 

P ro Ja u ( 7 ) 

The vector W 2 is called the vector component of u orthogonal to a. Since we have W 2 = 11 — wj, this vector can be written in 
notation 7 as 

w 2 = u - proj a ii 




Figure 3.3.6 


The vector u is the sum of wi and W 2 , where wi is parallel to a and W 2 is perpendicular to a. 










The following theorem gives formulas for calculating proj a ii and u — proj a ii- 


THEOREM 3.3.3 


If u and a are vectors in 2-space or 3-space and if ^ Q, then 


proj a ii = 11 *1, a (vector component of u along a ) 

Hair 


n — proj a u = u — 


u - a 


a (vector component u orthogonal to a) 


Let Wl — proj fl n and W 2 = u — proj a u. Since w\ is parallel to a, it must be a scalar multiple of a, so it can be written in 
the form Wl = Thus 


u — wi -I- W2 = £a 4- W2 


( 8 ) 


Taking the dot product of both sides of 8 with a and using Theorems Theorem 3.3.1a and Theorem 
3.3.2 yields 


u ■ a = (£a + w' 2 ) - a = £||a|| J 4- W 2 - a 


(9) 


But W2 - a = 0 since W 2 is perpendicular to a; so 9 yields 


k = 


u - a 


Since proj a ii = wi = k-n, we obtain 


proj a ii = -^~^-a 
Hall 2 


Vector Component of u Along a 


Let u = (2, — 1, 3) and a = (4, -1,2)- Find the vector component of u along a and the vector component of u orthogonal to 
a. 


Solution 


u - a = (2) (4) + ( — 1) ( — 1) + (3) (2) = 15 
I a || 2 = 4 2 4 ( — l) 2 -P 2 2 = 21 


Thus the vector component of u along a is 


proj a u = -^a = lf(4, - 1, 2) = (f, - f, f) 


and the vector component of u orthogonal to a is 










n — proj a ii = (2, - 1, 3) - (y, - 

As a check, the reader may wish to verify that the vectors u _ proj a ii 
is zero. 


5 10\ _ / 6 _ 2 in 
7 ’ 1 { 7’ 7’ 7 } 

and a are perpendicular by showing that their dot product 


♦ 


A formula for the length of the vector component of u along a can be obtained by writing 


llpr ° Ja ' i || = 


ii - a 


11*11 
u - a 


11*11 
In - a I 


Formula (5) of Section 3.2 


Sincellair >0 


which yields 


IIP™j a «ll = |j^j| 

lf.0 denotes the angle between u and a, then u - a = ||u|| ||a|| cos 9, so 10 can also be written as 

Hp ro Ja ll ll = IMI| cos 0| 


(Verify.) A geometric interpretation of this result is given in Figure 3.3.7. 



( 10 ) 


(ID 



( b ) ^ < ft < TT 


Figure 3.3.7 


As an example, we will use vector methods to derive a formula for the distance from a point in the plane to a line. 


Distance Between a Point and a Line 


Find a formula for the distance D between point P[\(xq, yp) and the line ax V by + c = 0- 


Solution 



















Let Q(x\,yi) be any point on the line, and position the vector n = (a, b) so that its initial point is at Q. 


By virtue of Example 5, the vector n is perpendicular to the line (Figure 3.3.8). As indicated in the figure, the distance D is 
equal to the length of the orthogonal projection of Qp^ on n; thus, from 10, 


£= HproJnC^oll = 


Q ?0 -n 

INT 


But 

QF\ o - (xo-^i,70“7i) 

QP 0 -n = fl(io-ii) + iO'o-^l) 
iinii = b 2 

so 

D= N*o-*i) +^0o-7i)| 
l la 2 + b 2 

Since the point g(xi, y i) lies on the line, its coordinates satisfy the equation of the line, so 

ax\+by\+c = Q or c——ax\^by\ 
Substituting this expression in 12 yields the formula 

|fljr 0 + fop-I e| 

{Jw 


( 12 ) 


(13) 



Using the Distance Formula 


It follows from Formula 13 that the distance D from the point (1, — 2) to the line 2>x 4 4y — 6 = 0 is 

|(3)(l) + 4( — 2) — 6| _ | - 11| _n 
l/3 2 + 4 2 


D= ■ 


/25 


Exercise Set 3.3 



















Find u. v- 


1 . 

(a) u= (2, 3), y= (5, -7) 

(b) u = (-6, -2), v= (4, 0) 

(c) u= (1. — 5, )4, v = (3, 3, 3) 

(d) u = ( — 2, 2, 3), v= (1, 7, -4) 

In each part of Exercise 1, find the cosine of the angle 0 between u and v. 

2 . 

Determine whether u and v make an acute angle, make an obtuse angle, or are orthogonal. 

3. 

(a) u= (6, 1, 4)* v= (2, 0, -3) 

(b) u= (0,0, -l),v=(l, 1, 1) 

(c) u=(_6, 0,4),v=(3, 1,6) 

(d) u =(2,4, - 8), v =(5,3,7) 

Find the orthogonal projection of u on a. 

4 

(a) u= (6, 2), a = (3, -9) 

(b) u = (- 1, —2), a = ( — 2, 3) 

(c) u= (3, 1, -7), a = (1,0, 5) 

(d) u= (1,0,0), a = (4,3, 3) 

In each part of Exercise 4, find the vector component of u orthogonal to a. 


5. 



In each part, find ||proj a u||- 


(a) u= (1, -2),a = (-4, -3) 

(b) u= (5, 6), a= (2, - 1) 

(c) u= (3, 0,4), a =(2, 3, 3) 

(d) u=(3. — 2, 6), a = (1, 2, -7) 

Let u = (5 > _ 2, 1), v = (1, 6, 3), and k — — 4- Verify Theorem 3.3.2 for these quantities. 

7 . 

8 . 

(a) Show that v = (a, b) and w — (—b,a) are orthogonal vectors. 

(b) Use the result in part(a) to find two vectors that are orthogonal to v = (2, — 3). 

(c) Find two unit vectors that are orthogonal to ( _ 2, 4). 

Let u = (3 4), y — (5 — 1), and w = (7 1). Evaluate the expressions. 

9. ' ' ' 

(a) u - (7v + w) 

(b) || (u - w)w|| 

(c) ||u||(v-w) 

(d) (||u||v) w 

Find five different nonzero vectors that are orthogonal to u = (5 — 2 3) • 

10 . 

Use vectors to find the cosines of the interior angles of the triangle with vertices (0, — 1), (1, — 2), and (4 j 1). 


12. 


Show that A (3, 0, 2), B (4, 3, 0), and C (8, 1, — 1) are vertices of a right triangle. At which vertex is the right angle? 


13 . 


Find a unit vector that is orthogonal to both u = (1, 0, 1) and v = (0, 1, !)• 


A vector a in the xy-plane has a length of 9 units and points in a direction that is 120° counterclockwise from the positive 

14. x-axis, and a vector b in that plane has a length of 5 units and points in the positive y-di recti on. Find a . b- 

A vector a in the xy-plane points in a direction that is 47° counterclockwise from the positive x-axis, and a vector b in that 

15. plane points in a direction that is 43° clockwise from the positive x-axis. What can you say about the value of a . b? 


Let p = (2, £) and q = (3, 5). Find k such that 


(a) p and q are parallel 


(b) p and q are orthogonal 


(c) the angle between p and q is 7i/3 


(d) the angle between p and q is 7t/4 


17 . 


Use Formula 13 to calculate the distance between the point and the line. 


(a) 4x A 3y + 4 = 0; ( — 3, 1) 


(b) y = _4xH 2; (2, -5) 


(c) 3x -I- y = 5; (1, 8) 


Establish the identity ||u -\ v|| 2 -I- ||u - v|| 2 = 2||u|| 2 4- 2||v|| 2 . 


Establish the identity u - v = i ||u + v|| 2 — j ||u — v|| 2 . 


20 . 


Find the angle between a diagonal of a cube and one of its faces. 


21. 


Let i,j, and k be unit vectors along the positive x, y, and z axes of a rectangular coordinate system in 3-space. If 
v = (a, b, c) is a nonzero vector, then the angles a, p, and y between v and the vectors i,j, and k, respectively, are called 
the direction angles of v (see accompanying figure), and the numbers cos a, cos p, and cos y are called the direction 


cosines of v. 


(a) Show that cos a — a! ||v 

(b) Find cos P and cos y. 


(c) Show that y / 1 |y|| = (cos a, cos /?, cos 7 ) 

(d) Show that cos^a+ cos^/3 4 - cos^o = 1. 


Figure Ex-21 

Use the result in Exercise 21 to estimate, to the nearest degree, the angles that a diagonal of a box with dimensions 10 cm 

22. x 15 cm x 25 cm makes with the edges of the box. 

Note A calculator is needed. 

Referring to Exercise 21, show that two nonzero vectors, yj and v r 2 , in 3-space are perpendicular if and only if their 

23. direction cosines satisfy 

cosq.’icosq '2 -I- cos,3icos/92 4- COS 71 COS 72 = 0 



(a) Find the area of the triangle with vertices A(2, 3), C(4, 7), and D( _ 5 , 8 ). 

(b) Find the coordinates of the point B such that the quadrilateral ABCD is a parallelogram. What is the area of this 
parallelogram? 


Show that if v is orthogonal to both wj and W 2 , then v is orthogonal to £ lWl -|- k-jy^'2 f° r scalars k\ and kj- 


Let u and v be nonzero vectors in 2- or 3-space, and let k — ||u|| and l — ||y||. Show that the vector w = iu 1- kv bisects the 
26. angle between u and v. 







Discussion 

Discovery 


In each part, something is wrong with the expression. What? 


27 . 

(a) u - (v - w) 

(b) (u - v) + w 

(c) ll«-v|| 

(d) k - (u+ v) 

Is it possible to have proj a ii = proj u a? Explain your reasoning. 

28 . 

If u * 0, is it valid to cancel u from both sides of the equation u . v — u . w and conclude that 

29. v = w? Explain your reasoning. 

Suppose that u, v, and w are mutually orthogonal nonzero vectors in 3-space, and suppose that 

30. you know the dot products of these vectors with a vector r in 3-space. Find an expression for r in 
terms of u,v, w, and the dot products. 

Hint Look for an expression of the form r = cqu 4 - c-jy f- c 3 w. 

Suppose that u and v are orthogonal vectors in 2-space or 3-space. What famous theorem is 
31* described by the equation ||u + v || 2 = ||u|| 2 + ||v|| 2 ? Draw a picture to support your answer. 
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O A In many applications of vectors to problems in geometry, physics, and 

engineering, it is of interest to construct a vector in 3-space that is perpendicular 
CROSS PRODUCT to two given vectors. In this section we shall show how to do this. 


Cross Product of Vectors 

Recall from Section 3.3 that the dot product of two vectors in 2-space or 3-space produces a scalar. We will now define a type of 
vector multiplication that produces a vector as the product but that is applicable only in 3-space. 


DEFINITION 


If u = {u\, u'i, U'i) and v = (vi, V 2 , V 3 ) are vectors in 3-space, then the cross product u x v is the vector defined by 

UXV = (u 2 v 3 — U 2 V 2> «3V1 -UIV 3 , u\V2 -U2Vl) 

or, in determinant notation, 


u x v = 


U2 u-2 
v 2 v 3 


u 1 u 3 
VI v 3 


u\ U2 
vi v 2 


( 1 ) 


Remark Instead of memorizing 1, you can obtain the components of u x v as follows: 


Form the 2x3 matrix 
components of v. 


«1 

vi 


u 2 

v 2 


“ 3 
v 3 


whose first row contains the components of u and whose second row contains the 


To find the first component of u x v, delete the first column and take the determinant; to find the second component, delete 
the second column and take the negative of the determinant; and to find the third component, delete the third column and take 
the determinant. 


Calculating a Cross Product 


Find ii x v> where u = (1, 2, — 2) and v = (3, 0, !)• 


Solution 


From either 1 or the mnemonic in the preceding remark, we have 



1 -2 
3 1 


1 

3 


2 

0 


= (2, -7, - 6 ) 



























There is an important difference between the dot product and cross product of two vectors—the dot product is a scalar and the 
cross product is a vector. The following theorem gives some important relationships between the dot product and cross product 
and also shows that u x v is orthogonal to both u and v. 


THEOREM 3.4.1 

Relationships Involving Cross Product and Dot Product 

If u, v, and w are vectors in 3-space, then 

(a) u - (u x v) = 0 (u x v is orthogonal to u) 

(b) v - (u x v) = 0 (u x v is orthogonal to v) 

(c) ||u xT|| 2 = |]n|| 2 ||v|| 2 — (u- v) 2 (Lagrange’s identity) 

(d) u x (v x w) = (u - w)v — (u ■ v)w (relationship between cross and dot products) 

(e) (u x v) x w = (u - w)v — (v ■ w)u (relationsliip between cross and dot products) 


(a) Let u = (u!, a 2 , uj) and v= (v 1 .v 2 .v 3 ). Then 


u - (u x v) = (u 1 , a 2 > uf) - (u 2 v 3 — U 3 V 2 , U 3 V 1 — U 1 V 3 , u\V 2 — u 2 v l) 

= u 1 2 v 3 — y 3 v 2) 4 2^2(“3^1 -U1V3) + «3(uiv 2 -u 2 vi) = 0 


Proof (b) Similar to (a). 


Proof (c) Since 


ll x v 


I 2 = (222V3 — W3V2) 2 


(223V1 -u\V2) 2 + (uiv 2 - w 2 vi) 2 


and 


||u|| 2 ||v || 2 - (u- v ) 2 = {u\ 3- uj 4- u|)(v 2 + v\ + v 2 ) - (aIV! + u 2 v 2 V M 3 V 3) 2 


( 2 ) 

( 3 ) 


the proof can be completed by "multiplying out" the right sides of 2 and 3 and verifying their equality. 



Proof (d) and (e) See Exercises 26 and 27. 



Joseph Louis Lagrange (1736-1813) was a French-Italian mathematician and astronomer. Although his father wanted him to 
become a lawyer, Lagrange was attracted to mathematics and astronomy after reading a memoir by the astronomer Halley. At 
age 16 he began to study mathematics on his own and by age 19 was appointed to a professorship at the Royal Artillery School 
inTurin. The following year he solved some famous problems using new methods that eventually blossomed into a branch of 
mathematics called the calculus of variations . These methods and Lagrange's applications of them to problems in celestial 
mechanics were so monumental that by age 25 he was regarded by many of his contemporaries as the greatest living 
mathematician. One of Lagrange's most famous works is a memoir, Meecanique Analytique, in which he reduced the theory of 
mechanics to a few general formulas from which all other necessary equations could be derived. 


Napoleon was a great admirer of Lagrange and showered him with many honors. In spite of his fame, Lagrange was a shy and 
modest man. On his death, he was buried with honor in the Pantheon. 


u x v Is Perpendicular to u and to v 


Consider the vectors 

In Example 1 we showed that 

Since 


u= (1,2, -2) and v=(3,0, 1) 
u x v = (2, — 7, — 6 ) 


u- (ux v) = (1) (2) + (2)( — 7) + ( — 2)( — 6) = 0 
and 

V - (u X v) = (3) (2) +■ (0) ( — 7) 4- ( 1) ( — 6) = 0 
u x v is orthogonal to both u and v, as guaranteed by Theorem 3.4.1. 

The main arithmetic properties of the cross product are listed in the next theorem. 


♦ 


THEOREM 3.4.2 




Properties of Cross Product 

Ifu, v, and w are any vectors in 3-space and k is any scalar, then 

(a) u x v = — (v x u) 

(b) u x (v f- w) = (u x v) 4- (u x w) 

(c) (u 4 v) x w = (u x w) 4- (v x w) 

(d) A:(ii x v) = (An) x v = u x (&v) 

(e) ux0 = 0xu = 0 

(f) u x u = 0 

The proofs follow immediately from Formula 1 and properties of determinants; for example, (a) can be proved as follows: 


Proof (a) Interchanging u and v in 1 interchanges the rows of the three determinants on the right side of 1 and hence changes the 
sign of each component in the cross product. Thus u x v = — (v x u)- 


The proofs of the remaining parts are left as exercises. 


EXAMPLE 3 Standard Unit Vectors 


Consider the vectors 


i= (1,0,0), j= (0,1,0), k= (0,0,1) 


These vectors each have length 1 and lie along the coordinate axes (Figure 3.4.1). They are called the standard unit vectors in 
3-space. Every vector v = (v i, v^, vj) in 3-space is expressible in terms of i,j, and k since we can write 

v= (vi, v% v?) =vi(l, 0, 0) 4- v^(0, 1, 0) 4- v^(0, 0, 1) = vii4- vii + v^k 


For example. 


From 1 we obtain 


(2, — 3, 4) = 2i — 3j 4- 4k 


/ 0 0 
l 1 0 


1 0 
0 0 


1 0 
0 1 


ix j = 


= ( 0 , 0 , l)=k 











Figure 3.4.1 


The standard unit vectors. 


♦ 


The reader should have no trouble obtaining the following results: 

ixi = 0 jxj = 0 kxk = 0 

ixj = k jxk =i kxi = j 

jxi= —k k x j= —i ixk= —j 

Figure 3.4.2 is helpful for remembering these results. Referring to this diagram, the cross product of two consecutive vectors going 
clockwise is the next vector around, and the cross product of two consecutive vectors going counterclockwise is the negative of the 
next vector around. 



Figure 3.4.2 


Determinant Form of Cross Product 


It is also worth noting that a cross product can be represented symbolically in the form of a formal 3x3 determinant: 

i j k 


ll x v = 


U 1 «2 “3 
vi V 2 v 3 


U2 U2 
v 2 v 3 


l — 


u i U2 
vi v 3 


JF 


u\ U2 
vi v 2 


k 


For example, if u = (1, 2, - 2) and v = (3, 0, 1), then 


u x v = 


i j k 
12-2 
3 0 1 


= 2i —7j —6k 


which agrees with the result obtained in Example 1. 


(4) 


It is not true in general that u x (v x w) = (u x v) x w- For example. 


and 


ix(jxj)=ix0 = 0 


(ixj) xj = kxj = - i 


so 


i x (i x j) * (i x i) x j 

We know from Theorem 3.4.1 that u x v is orthogonal to both u and v. If u and v are nonzero vectors, it can be shown that the 
direction of u x v can be determined using the following “right-hand rule”* (Figure 3.4.3): Let 0 be the angle between u and v, and 
suppose u is rotated through the angle 0 until it coincides with v. If the fingers of the right hand are cupped so that they point in the 















direction of rotation, then the thumb indicates (roughly) the direction of u x v- 


UKV 



Figure 3.4.3 


The reader may find it instructive to practice this rule with the products 

i x j = k, j x k = i, k x i = j 


Geometric Interpretation of Cross Product 

If u and v are vectors in 3-space, then the norm of u x v has a useful geometric interpretation. Lagrange's identity, given in 
Theorem 3.4.1, states that 

||uxv|| 2 =||u|| 2 ||v|| 2 -(u-v) 2 (5) 

If 0 denotes the angle between u and v, then u ■ v = ||u|| ||v|| cos #, so 5 can be rewritten as 

||u x v|| 2 = ||u|| 2 ||v|| 2 — ||u|| 2 ||v|| 2 cos 2 # 

= IMI 2 l|v|| 2 (l - cos 2 #) 

= ||u|| 2 ||v|| 2 sm 2 # 

Since 0 < # < tt, it follows that sin # > 0, so this can be rewritten as 

||uxv|| = ||u||||v|| sin# 

But ||v|| sin# is the altitude of the parallelogram determined by u and v (Figure 3.4.4). Thus, from 6, the area A of this 
parallelogram is given by 

A= (base) (altitude) = ||u||||v|| sin#= ||uxv|| 



Figure 3.4.4 

This result is even correct if u and v are collinear, since the parallelogram determined by u and v has zero area and from 6 we have 
u x v = 0 because # = 0 in this case. Thus we have the following theorem. 


THEOREM 3.4.3 




Area of a Parallelogram 

Ifu and v are vectors in 3-space, then ||u x v|| is equal to the area of the parallelogram determined by u and v. 


XA Area of a Triangle 


Find the area of the triangle determined by the points (2, 2, 0)^ -P 2 C — 1, 0, 2)> and p 3 ( 0 , 4 , 3 ). 


Solution 


The area A of the triangle is ^ the area of the parallelogram determined by the vectors pypj anc * P\P 2 (Fig ure 3.4.5). Using the 
method discussed in Example 2 of Section 3.1, = ( — 3 — 2 2) anc * P\P 2 = ( — 2 2 3 )- It follows that 

1^3 = 010 , 5 , - 10 ) 


and consequently. 


A = l\\P l P 2 xP l P 3 \\ = ±(\5)=^ 



DEFINITION 


If u, v, and w are vectors in 3-space, then 


u - (v x w) 

is called the scalar triple product of u. 

v, and w. 


The scalar triple product of u = ( Ul ? v = (vi, V 2 , v 3 ), and w=(wi,W 2 >™ 3 ) can b e calculated from the formula 


u - (v X w) = 


u\ u 2 2^3 

vi v 2 v 3 

W\ >i?2 >i ? 3 


(7) 


This follows from Formula 4 since 

























u - (v x w) = u ■ 


f v 2 v 3 

yw2 v>2 


vi 

Wl 



VI 

Wl 



v 2 

w 2 


v 3 


VI 

v 3 


VI 

V2 

w 3 

“1 - 

Wl 

w 3 

U2 + 


w>2 


«1 «2 “3 

vi V 2 v 3 
wi W2 w 3 


EXAMPLE 5 Calculating a Scalar Triple Product 


Calculate the scalar triple product u - (v x w) of the vectors 

u = 3i — 2j — 5k, v = i4 4j — 4k, w = 3j + 2k 

Solution 

From 7, 

11 - (v X w) 

= 60 + 4-15 = 49 


3 -2 -5 
1 4 -4 

0 3 2 


= 3 


4 -4 
3 2 


-(- 2 ) 


1 -4 

0 2 


+ C-5) 


1 4 
0 3 


Remark The symbol ( u . v ) x w makes no sense because we cannot form the cross product of a scalar and a vector. Thus no 
ambiguity arises if we write u . v x w rather than u - (v x w). However, for clarity we shall usually keep the parentheses. 


It follows from 7 that 


u - (v x w) = w - (u x v) = v - (w x ll) 


since the 3x3 determinants that represent these products can be obtained from one another by two row interchanges. (Verify.) 
These relationships can be remembered by moving the vectors u, v, and w clockwise around the vertices of the triangle in Figure 


3.4.6. 


II 



Figure 3.4.6 


Geometric Interpretation of Determinants 

The next theorem provides a useful geometric interpretation of 2 x 2 and 3x3 determinants. 


THEOREM 3.4.4 



























(a) The absolute value of the determinant 


is equal to the area of the parallelogram in 2-space determined by the vectors w=(u\,u 2 ) and 
v = (v, v 2 ). (See Figure 3.4.7a.) 




(b) The absolute value of the determinant 


det 


mi u 2 u 2 
vi v 2 v 3 

W\ W 2 


is equal to the volume of the parallelepiped in 3-space determined by the vectors u = (mi, u 2 , u 2 ), 
y — ( v b v 2 > v 3 )/ and w= (wi, w 2 , w 3 )- (See Figure 3.4.7b.) 


The key to the proof is to use Theorem 3.4.3. However, that theorem applies to vectors in 3-space, whereas \\=(u\ r u 2 ) 
and v = (vi, v 2 ) are vectors in 2-space. To circumvent this “dimension problem,” we shall view u and v as vectors in the xy -plane 
of an xyz-coordinate system (Figure 3.4.8a), in which case these vectors are expressed as u = ( M1j U2 , 0) and v = (v\, v 2 , 0)- Thus 


uxv = 


i j k 

u 2 0 
vi v 3 0 


w l u 2 
vi v 2 


k = det 


w i u 2 
vi v 2 




















Figure 3.4.8 

It now follows from Theorem 3.4.3 and the fact that ||k|| = 1 that the area A of the parallelogram determined by u and v is 


A = ||u x v|| = ||det 


u l U2 
vi v 2 


k|| = 


det 


a l U2 
vi v 2 


l|k|| = 


det 


u\ U2 
vi v 2 


which completes the proof. 


Proof (b) As shown in Figure 3.4.8/?, take the base of the parallelepiped determined by u, v, and w to be the parallelogram 
determined by v and w. It follows from Theorem 3.4.3 that the area of the base is ||v x w|| and, as illustrated in Figure 3.4.8 b, the 
height h of the parallelepiped is the length of the orthogonal projection of u on v x w- Therefore, by Formula 10 of Section 3.3, 


A - llprojvxwnll - ■ 

It follows that the volume V of the parallelepiped is 

V = (area of base) - height = || v x w 

so from 7, 


|u - (y x w) | 
llv x wll 


|u-(vxw)| 

li- = \ n - (v x w) 

VXW I I 




'^1 

“2 

u 2 1 

v= 

det 

VI 

v 2 

v 3 



Wl 

w 2 

M?3 


which completes the proof. 


Remark If V denotes the volume of the parallelepiped determined by vectors u, v, and w, then it follows from Theorem 3.3 and 
Formula 7 that 



volume of parallelepiped 
determined by u, v, andw 


|u - (v x w) | 


( 8 ) 


From this and Theorem 3.3.1 b, we can conclude that 

u - (v x w) = =t V 

where the + or - results depending on whether u makes an acute or an obtuse angle with v x w- 

Formula 8 leads to a useful test for ascertaining whether three given vectors lie in the same plane. Since three vectors not in the 
same plane determine a parallelepiped of positive volume, it follows from 8 that |u - (v x w) | = 0 if and only if the vectors u, v, and 
w lie in the same plane. Thus we have the following result. 


THEOREM 3.4.5 


If the vectors u = (u\, u 2 , w 3 )> v = (vi, v 2 , v 3 ), and w = (wp w 2 . 

w 3 ) 

have the same initial point, then they lie in the same 

plane if and only if 






Ul 

u 2 

u 3 


u - (v X w) = 

VI 

v 2 

v 3 

= 0 


Wl 

W2 

w 3 



Independence of Cross Product and Coordinates 

Initially, we defined a vector to be a directed line segment or arrow in 2-space or 3-space; coordinate systems and components 
were introduced later in order to simplify computations with vectors. Thus, a vector has a “mathematical existence” regardless of 























whether a coordinate system has been introduced. Further, the components of a vector are not determined by the vector alone; they 
depend as well on the coordinate system chosen. For example, in Figure 3.4.9 we have indicated a fixed vector v in the plane and 
two different coordinate systems. In the xjy-coordinate system the components of v are (1, 1), and in the x’y ''-system they are 




This raises an important question about our definition of cross product. Since we defined the cross product u x v in terms of the 
components of u and v, and since these components depend on the coordinate system chosen, it seems possible that two fixed 
vectors u and v might have different cross products in different coordinate systems. Fortunately, this is not the case. To see that this 
is so, we need only recall that 


u x v is perpendicular to both u and v. 


The orientation of « x v is determined by the right-hand rule. 


UX V = 11 


v|| sin 0. 


These three properties completely determine the vector u x y: the first and second properties determine the direction, and the third 
property determines the length. Since these properties of u x y depend only on the lengths and relative positions of u and v and not 
on the particular right-hand coordinate system being used, the vector u x y will remain unchanged if a different right-hand 
coordinate system is introduced. We say that the definition of u x v is coordinate free. This result is of importance to physicists 
and engineers who often work with many coordinate systems in the same problem. 


u x v Is Independent of the Coordinate System 


Consider two perpendicular vectors u and v, each of length 1 (Figure 3.4.10a). If we introduce an xjvz-coordinate system as shown 
in Figure 3.4.10Z?, then 

u= (1,0, 0)=i and v=(0, 1, 0) = j 


so that 


u x v = i x j = k = (0, 0, 1) 

However, if we introduce an x>V-coordinate system as shown in Figure 3.4.10c, then 

u= (0, 0, 1) =k and v=(l,0, 0)=i 


so that 


uxv = kxi = j= (0, 1, 0) 


But it is clear from Figures 3.4.10Z? and 3.4.10c that the vector (0, 0, 1) in the xjyz-system is the same as the vector (0, 1, 0) in the 
x'^V-system. Thus we obtain the same vector u x v whether we compute with coordinates from the xyz- system or with 
coordinates from the x^-V-system. 
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Exercise Set 3.4 

Click here for Just Ask! 

Let u = (3, 2, — 1)> v = (0, 2, - 3), and w = (2, 6, !)■ Compute 

(a) vxw 

(b) u x (v x w) 

(c) (u x v) x w 

(d) (u x v) x (v x w) 

(e) u x (v - 2w) 

(f) (u x v) — 2iv 

Find a vector that is orthogonal to both u and v. 

2 . 

(a) u = ( — 6, 4, 2)> v= (3, 1, 5) 

(b) u = ( — 2, 1, 5),v= (3, 0, -3) 

Find the area of the parallelogram determined by u and 

3 . 




f \ 


•s 


U :-L \ 




Ut) 

Figure 3.4.10 


t/>) 


(a) u= (1, — 1, 2), v= (0, 3, 1) 














(b) u= (2, 3, 0),v= (- 1, 2, -2) 

(c) u= (3, — 1, 4)> v= (6, -2,8) 

Find the area of the triangle having vertices P, Q, and R. 

4 

(a) P(2,6, -1), (2(1,1, 1), *(4,6,2) 

(b) P( 1, -1,2), 0(0, 3, 4), *(6,1,8) 


Verify parts (a), (b ), and (c) of Theorem 3.4.1 for the vectors u = (4 j 2, 1) and v = (— 3, 2,7). 


Verify parts (a), (b), and (c) of Theorem 3.4.2 for u = (5, - 1, 2), v = (6, 0, - 2), and w = (1, 2, - 1). 

6 . 


Find a vector v that is orthogonal to the vector u = ( 

7. 


Find the scalar triple product u . ( v x w)- 

8 . 


(a) u = ( — 1, 2, 4). v= (3, 4, - 2), w= ( - 1, 2, 5) 


(b) u= (3, -l,6),v=(2,4,3),w=(5, -1,2) 


Suppose that u . ( v x w) = 3- Find 

9. 


(a) u - (wxv) 


(b) (v x w) - u 


(c) W • (u X v) 


(d) v - (u x w) 


(e) (uxw) ■ v 


(f) v - (wxw) 





Find the volume of the parallelepiped with sides u, v, and w. 

10 . 


(a) u= (2, -6, 2), v= (0,4, -2),w=(2, 2, -4)) 


(b) u= (3, 1, 2). v= (4, 5, 1)> w= (1, 2, 4) 


Determine whether u, v, and w lie in the same plane when positioned so that their initial points coincide. 

11 . 


(a) u = (-1, — 2, 1)> v = (3, 0, -2),w=(5, -4,0) 


(b) u= (5, -2, l),v= (4, - 1, l),w= (1, - 1, 0) 


(c) u= (4, -8, l),v=(2, 1, — 2)> w= (3, -4, 12) 


12 . 


Find all unit vectors parallel to the yz-plane that are perpendicular to the vector (3 ; _ ] 2)- 


Find all unit vectors in the plane determined by u = (3 ; o, 1) and v = (1, —1,1) that are perpendicular to the vector 
13 - w= (1, 2, 0). 

Let a 133)? b = (Z?i, b2, £3)’ c = (c 1, C2 , ^3)5 an d <1 = (d\, d2, d^)- Show that 

(a -h cl) - (b x c) = a - (b x c) + cl - (b x c) 

Simplify (u _|_ v ) x (u — v) • 

15 « 

Use the cross product to find the sine of the angle between the vectors u = (2, 3, — 6) and v = (2,3, 6). 


17. 

(a) Find the area of the triangle having vertices j 4(1 ; 0, 1). 5(0, 2, 3), and C(2, 1, 0)- 

(b) Use the result of part (a) to find the length of the altitude from vertex C to side AB- 


Show that if u is a vector from any point on a line to a point P not on the line, and v is a vector parallel to the line, then the 

18. distance between P and the line is given by ||u x v|| / ||v||. 

Use the result of Exercise 18 to find the distance between the point P and the line through the points A and 

19. 


(a) P(-3, 1,2M(1, l,0),5(-2,3, -4) 





(b) P{4, 3, 0), A(2, 1, -3), B(0, 2, - 1) 


Prove: If o is the angle between u and v and u . v * 0, then tan 9= ||u x v|| / (u - v)- 

20. 

Consider the parallelepiped with sides u = (3, 2, 1)> v = (1, 1, 2), and w = (1, 3, 3)- 

21. 

(a) Find the area of the face determined by u and w. 

(b) Find the angle between u and the plane containing the face determined by v and w. 

Note The angle between a vector and a plane is defined to be the complement of the angle 0 between the vector and 
that normal to the plane for which 0 < 0 < tt / 2. 

Find a vector n that is perpendicular to the plane determined by the points ^(0, —2, 1), 5(1, —1, — 2)- and C( — 1, 1, 0)- 
22* [See the note in Exercise 21.] 

Let m and n be vectors whose components in the xyz-system of Figure 3.4.10 are m = (0 0 1) and n = (0 1 0)- 

23. 

(a) Find the components of m and n in the x'y'J -\system of Figure 3.4.10. 

(b) Compute m x n using the components in the xyz-system. 

(c) Compute m x n using the components in the v'VV- system. 

(d) Show that the vectors obtained in (b) and (c) are the same. 

Prove the following identities. 

24. 

(a) (u -i- ,tv) x v = u x v 

(b) u - (v x z) = — (u x z) - v 

Let u, v, and w be nonzero vectors in 3-space with the same initial point, but such that no two of them are collinear. Show that 

25. 

(a) u x (v x w) lies in the plane determined by v and w 

(b) (u x v) x w lies in the plane determined by u and v 

Prove part (cl) of Theorem 3.4.1. 

26. 



Hint First prove the result in the case where w = i = (0, 0, 1) then when w = j = ( 0 , 1,0), and then when w = k. = (0, 0, 1)- 
Finally, prove it for an arbitrary vector w = (w i, W 2 , W 3 ) by writing w = w ji -I- W 2 j 4 - W 3 L 


Prove part ( e ) of Theorem 3.4.1. 

27. 

Hint Apply part (a) of Theorem 3.4.2 to the result in part (cl) of Theorem 3.4.1. 


Let 11 = (l ; Z, — 1), v = (1, 1, 2), and w = (3, — 1, 2)- Calculate u x (v x w) using the technique of Exercise 26; then check 
your result by calculating directly. 

Prove: If a, b, c, and d lie in the same plane, then (a x b) x (c x d) = 0- 


30. 


It is a theorem of solid geometry that the volume of a tetrahedron is j(area of base) - (height). Use this result to prove that the 
volume of a tetrahedron whose sides are the vectors a, b, and c is ^-|a - (b x c) | (see the accompanying figure). 



Use the result of Exercise 30 to find the volume of the tetrahedron with vertices P, Q, R, S. 

31. 

(a) P{- 1,2, 0), Q(2, 1, - 3), R( 1, 0, 1), S( 3, - 2, 3) 

(b) P{ 0 , 0, 0), g(l, 2, - 1), R{ 3, 4, 0), £(-!,- 3, 4) 


Prove part ( b ) of Theorem 3.4.2. 

32. 


Prove parts (c) and (cl) of Theorem 3.4.2. 

33. 


Prove parts ( e ) and (j) of Theorem 3.4.2. 

34. 


Discussion 
Discovery 35 

(a) Suppose that u and v are noncollinear vectors with their initial points at the origin in 3-space 
Make a sketch that illustrates how w = v x (11 x v) is oriented in relation to u and v. 




(b) For w as in part (a), what can you say about the values of u - w and y - w? Explain your 
reasoning. 


If u * 0. is it valid to cancel u from both sides of the equation u x v = u x w and conclude that v = w 
36. ? Explain your reasoning. 

Something is wrong with one of the following expressions. Which one is it and what is wrong? 

37 

" U - (V X w), 11X V X W, (ll - v) X w 

What can you say about the vectors u and v if u x v = 0? 

38. 

Give some examples of algebraic rules that hold for multiplication of real numbers but not for the 

39. cross product of vectors. 
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3.5 

LINES AND PLANES IN 
3-SPACE 


In this section we shall use vectors to derive equations of lines and planes in 
3-space. We shall then use these equations to solve some basic geometric 
problems. 


Planes in 3-Space 

In analytic geometry a line in 2-space can be specified by giving its slope and one of its points. Similarly, one can specify a 
plane in 3-space by giving its inclination and specifying one of its points. A convenient method for describing the inclination of 
a plane is to specify a nonzero vector, called a normal, that is perpendicular to the plane. 

Suppose that we want to find the equation of the plane passing through the point y$, zq) and having the nonzero vector 

n = (a, b, c) as a normal. It is evident from Figure 3.5.1 that the plane consists precisely of those points p(x, y, z") for which the 
vector p^p is orthogonal to //; that is, 

n-.P^=0 

Since p fj p = _ XQ y —y$ z — zq)’ Equation 1 can be written as 

a(x -*o) -I b(y - 70 ) + c(z-zq) — 0 ^ 2 ) 

We call this the point-normal form of the equation of a plane. 




Figure 3.5.1 

Plane with normal vector. 


Finding the Point-Normal Equation of a Plane 


Find an equation of the plane passing through the point ( 3 ; _ \ s 7 ) and perpendicular to the vector n = ( 4 j 2 , — 5 ). 


Solution 


From 2 a point-normal form is 


4(* — 3) + 2(y + 1) — 5(z — 7) = 0 








By multiplying out and collecting terms, we can rewrite 2 in the form 

ax 4 - by 4 - cz 4 - d = 0 

where a, /?, c, and d are constants, and a, b, and c are not all zero. For example, the equation in Example 1 can be rewritten as 

4x 4 - 2.y — 5z 4 - 25 = 0 

As the next theorem shows, planes in 3-space are represented by equations of the form ax + by -\- cz y d = 0- 


THEOREM 3.5.1 

If a, b, c, and d are constants and a, b, and c are not all zero, then the graph of the equation 

ax + by + cz + d = 0 

is a plane having the vector n= (a,b,c) as a normal. 

Equation 3 is a linear equation in x, y, and z\ it is called the general form of the equation of a plane. 


Proof By hypothesis, the coefficients a, b, and c are not all zero. Assume, for the moment, that a * 0- Then the equation 
ax + by \- cz 4 - d = 0 can be rewritten in the form a(x 4 - (d ! a)) -\- by H cz = 0- But this is a point-normal form of the plane 
passing through the point ( — d/ a, 0,0) and having n = (a, b, c) as a normal. 

If a = 0> then either £ * 0 or c * 0- A straightforward modification of the above argument will handle these other cases. 

I 

Just as the solutions of a system of linear equations 

ax 4 by = Aq 
cx -V dy = £2 

correspond to points of intersection of the lines ax by = k\ and cx dy = £2 i n the xy -plane, so the solutions of a system 

ax I by •+ cz = k\ 

dx-Vey + fz = k 2 ( 4 ) 

gx + hy \ iz = £3 

correspond to the points of intersection of the three planes ax -h by + cz = Aq, dx sy + f z — ky> and gx p hy + iz = ky 
In Figure 3.5.2 we have illustrated the geometric possibilities that occur when 4 has zero, one, or infinitely many solutions. 







(u) 




(<■’) 


if) 0 ?) W 


Figure 3.5.2 

(a) No solutions (3 parallel planes), (b) No solutions (2 parallel planes), (c) No solutions (3 planes with no 
common intersection), (d) Infinitely many solutions (3 coincident planes). (<?) Infinitely many solutions (3 planes 
intersecting in a line). (/) One solution (3 planes intersecting at a point), (g) No solutions (2 coincident planes 
parallel to a third plane). ( h ) In.nitely many solutions (2 coincident planes intersecting a third plane). 


Equation of a Plane Through Three Points 


Find the equation of the plane passing through the points ( 1 , 2, — 1 ), p 2 (2, 3, 1), and P^(3, — 1 , 2)- 


Solution 

Since the three points lie in the plane, their coordinates must satisfy the general equation ax -F by f- cz \- d = 0 of the plane. 
Thus 

a \- 2b — c d = 0 
2a \- 3b + c d = 0 
3a— b -\-2c -\- d = Q 

Solving this system gives a = — b = — c = j-t, d = t- Letting t = — 16, for example, yields the desired equation 

9x -I y — 5z— 16 = 0 

We note that any other choice of t gives a multiple of this equation, so that any value of t * 0 would also give a valid equation of 
the plane. 

Alternative Solution 


Since the points Pj (l ; 2 , — 1 )> P 2 Q, 3, 1 )> and P-${3, — 1, 2 ) lie in the plane, the vectors p^p 2 = (1 1 2 ) an< ^ 


PlP 3 =(2, -3,3) 


ait paianti 


is perpendicular to both and From this and the fact that Pj lies in the plane, a point-normal form for the equation of 
the plane is 



















9(x-l)+O-2)-5(z+l) = 0 9* H-.y - 5z-16 = 0 


♦ 


Vector Form of Equation of a Plane 


Vector notation provides a useful alternative way of writing the point-normal form of the equation of a plane: Referring to 
Figure 3.5.3, let r = ( x , y, z ) be the vector from the origin to the point p( x , y, z), let r 0 = (^■q i j^q i zq) be the vector from the 

origin to the point V 0 > z o)’ anc ' l et n = {a, b, c ) be a vector normal to the plane. Then p^p _ r _ r[| , so Formula 1 can be 

rewritten as 

n - (r - ro) = 0 (5) 

This is called the vector form of the equation of a plane. 


r 



V 

Figure 3.5.3 


Vector Equation of a Plane Using 5 


The equation 


(- 1, 2, 5) - (x-6,y-3,z I 4) = 0 


is the vector equation of the plane that passes through the point ( 6 , 1, — 4 ) and is perpendicular to the vector n = ( — 1 , 2 , 5 )- 

♦ 


Lines in 3-Space 

We shall now show how to obtain equations for lines in 3-space. Suppose that / is the line in 3-space through the point 
Pijfxij, vn, zq) and parallel to the nonzero vector v = (a, b, c) - It is clear (Figure 3.5.4) that / consists precisely of those points 
P(x, y, z) for which the vector pTp is parallel to v—that is, for which there is a scalar t such that 

P^P=tv (6) 


In terms of components, (6) can be written as 

(x - xq, y - yQ, z - zq) = (ta, tb, tc) 
from which it follows that * _ X q = ta, y - y o = tb, an d z - zq = tc , so 

X = XQ+ta, y=y$ptb, Z = ZQ \-tc 

As the parameter t varies from - oo to + M , the point p(x, y, z) traces out the line /. The equations 

x = xq \-ta, y =70 + lb, z = zq 4 tc ( — oo < t < + do ) 







are called parametric equations for l. 



Figure 3.5.4 

p^p is parallel to v. 


EXAf Parametric Equations of a Line 

The line through the point (1,2, — 3) and parallel to the vector v = (4, 5, _ 7) has parametric equations 

x = \ 4- 41, y = 2 4- 5t, z = — 3 — It (— oo<^< + oo) 


♦ 


Intersection of a Line and the xy-Plane 


(a) Find parametric equations for the line l passing through the points (2, 4,-1) and (5, 0, 7). 


(b) Where does the line intersect the xy-plane? 


Solution (a) 

Since the vector p^p 2 = (3 — 4 8) P ara ll e l to / and (2, 4,-1) lies on l, the line l is given by 

x = 2 + 31, y = 4 — At, z= — \ 4 St ( — oo<t< 4- oo) 


Solution (b) 

The line intersects the xy-plane at the point where z= — 1 4 St = 0, that is, where t = Substituting this value of t in the 

s 

parametric equations for l yields, as the point of intersection, 

(x,y,z) = (f, 1,0) 











Line of Intersection of Two Planes 


Find parametric equations for the line of intersection of the planes 

3x + 2y — 4z — 6 = 0 and x — 3y — 2z — 4 = 0 


Solution 


The line of intersection consists of all points (x,y,z) that satisfy the two equations in the system 


3x + 2y — 4z = 6 
x - 3y - 2z = 4 


Solving this system by Gaussian elimination gives x = yy- + jy-£, y = — yy- — ^t, z = t- Therefore, the line of intersection can be 
represented by the parametric equations 



z = t ( — oo < £ < + do ) 


♦ 


Vector Form of Equation of a Line 

Vector notation provides a useful alternative way of writing the parametric equations of a line: Referring to Figure 3.5.5, let 
r = y r z) be the vector from the origin to the point P(x,y,z), let r 0 = (.tq, .yri, zri) be the vector from the origin to the point 
‘Fo(*0> yo> z o)’ an< ^ v = (a, b, c ) be a vector parallel to the line. Then p^p _ r _ rQ , so Formula 6 can be rewritten as 

r - i’o = 

Taking into account the range of t-values, this can be rewritten as 

r = i’o — tv ( — do < t < 4- do ) ^ 

This is called the vector form of the equation of a line in 3-space. 


m. v, zi 



V 


Figure 3.5.5 


Vector interpretation of a line in 3-space. 


A Line Parallel to a Given Vector 









Problems Involving Distance 

We conclude this section by discussing two basic “distance problems” in 3-space: 

Problems 

(a) Find the distance between a point and a plane. 

(b) Find the distance between two parallel planes. 

The two problems are related. If we can find the distance between a point and a plane, then we can find the distance between 
parallel planes by computing the distance between either one of the planes and an arbitrary point p 0 in the other (Figure 3.5.6). 




Figure 3.5.6 

The distance between the parallel planes V and W is equal to the distance between and W. 

THEOREM 3.5.2 

Distance Between a Point and a Plane 

The distance D between a point ,Pg(*o> ^o> z o) an< ^ ^ e pl ane ax -\ by + cz + d = 0 is 


D |fljrp + byt] + czg + d\ 


(9) 



Let Q(x\,y\,z\) be any point in the plane. Position the normal n = (a, b, c ) so that its initial point is at Q. As illustrated 
in Figure 3.5.7, the distance D is equal to the length of the orthogonal projection of Qp^ on n. Thus, from (10) of Section 3.3, 


QP o •» 


P = llpr°Jn£^0 


li 


But 












QP 0 = (.XQ-xi,yo-yi,ZQ-zO 

Q?\ d -n = flCxo-xi) 4 b(ya-yi) -\-c(z a -zi) 


||n|| = j / fl 2 + £ 2 + c 2 


Thus 


D= K*o-*i)+&(ro-.yi) + c(zo-zi)| 

lfa 2 + i 2 + c 2 

Since the point Q(x\,y\,z\) lies in the plane, its coordinates satisfy the equation of the plane; thus 

ax\ 4 by i 4 cz\ 4 d = 0 


( 10 ) 


or 


d = — ax i — by i — cz\ 


Substituting this expression in (10) yields (9). 



Figure 3.5.7 

Distance from p 0 to plane. 


Note the similarity between (9) and the formula for the distance between a point and a line in 2-space [13 of Section 

3.3], 


Distance Between a Point and a Plane 


Find the distance D between the point (l ; _ 4 r _ 3) and the plane 2x — 3y 4 6z = — 1- 


Solution 


To apply (9), we first rewrite the equation of the plane in the form 

2x — 3y 4 6z 4 1 = 0 

Then 

|2(1)4( —3)( —4)46( —3) + l| 


1-31 


t/2 2 4 ( — 3) 2 H 6 2 


3 

7 


♦ 


Given two planes, either they intersect, in which case we can ask for their line of intersection, as in Example 6, or they are 
parallel, in which case we can ask for the distance between them. The following example illustrates the latter problem. 














Distance Between Parallel Planes 


The planes 


x + 2y — 2z = 3 and 2x A Ay — Az = 7 

are parallel since their normals, (1,2, — 2) and (2, 4, — 4), are parallel vectors. Find the distance between these planes. 


Solution 

To find the distance D between the planes, we may select an arbitrary point in one of the planes and compute its distance to the 
other plane. By setting y = z = 0 in the equation x A- 2y — 7z = 3, we obtain the point Pq( 3, 0, 0) in this plane. From (9), the 
distance between p Q and the plane 2x A- Ay — Az = 7 is 

D= |2(3) -I 4(0)4 (-4)(0)-7| _ i 
l/2 2 -|-4 2 4 (—4) 2 6 

# 


Exercise Set 3.5 



Click here for Just Ask! 


Find a point-normal form of the equation of the plane passing through P and having n as a normal. 

1 

(a) P( — 1, 3, -2); n= (-2, 1, -1) 

(b) P(l, 1,4); n= (1,9, 8) 

(c) P(2, 0, 0); n= (0, 0, 2) 

(d) p(0, 0,0); n= (1,2, 3) 


Write the equations of the planes in Exercise 1 in general form. 

2 . 

Find a point-normal form of the equations of the following planes. 

3. 

(a) — 3x A ly A- 2z = 10 

(b) 


x-Az = 0 







Find an equation for the plane passing through the given points. 

4. 

(a) P(- 4 , -1, — 1), Q( — 2, 0, 1), R( — 1, -2,-3) 

(b) P(5, A, 3), (2(4, 3, 1),J?(1,5.4) 

Determine whether the planes are parallel. 

5. 

(a) Ax —y A 2z = 5 and 7x — 3y A- Az = 8 

(b) x - Ay - 37 - 2 = 0 and 3* - \2y - 9z - 7 = 0 

(c) 2y = Sx — Az A- 5 and x = ^z A- jy 

Determine whether the line and plane are parallel. 

6 . 

(a) x — — 5 — At, y = 1 — t, z = 3 -|- 2t\ x A- 2y A- 3z — 9 = 0 

(b) x = 3t, y = 1 + 2t, z = 2 — 1\ Ax — y + 2z= 1 

Determine whether the planes are perpendicular. 

7. 

(a) 3x — y A z — 4 = 0>x4 2z=— 1 

(b) x-2yA 3z = A, - 2x A- 5y 4 Az= - 1 

Determine whether the line and plane are perpendicular. 

8 . 

(a) x = — 2 — At, y = 3 — 2t, z = 1 + 2t ; 2x I y —z = 5 

(b) x = 2. A- h y = 1 — t, z = 7 -l- 6x A- 6y — 1 = 0 

Find parametric equations for the line passing through P and parallel to 

9. 

(a) P{ 3, -1,2); n= (2, 1,3) 


(b) P( — 2, 3, -3); n= (6, -6,-2) 


(C) P{2, 2, 6); 11 = (0, 1, 0) 


(d) P(0,0,0);» = (1, -2,3) 


10 . 


Find parametric equations for the line passing through the given points. 


(a) (5, -2,4), (7,2, -4) 


(b) (0, 0, 0), (2, -1,-3) 


11 . 


Find parametric equations for the line of intersection of the given planes. 


(a) lx — 2y 4 3z = — 2 and -3x-\ y 2z I 5 = 0 


(b) 2x 4- 3y — 5z = 0 and y = 0 


12 . 


Find the vector form of the equation of the plane that passes through p Q and has normal n. 


(a) Pot-1,2, 4); n= (-2, 4, 1) 


(b) P 0 (2.0, -5);n=(-l,4,3) 


(c) P 0 (5, -2, l);n=(- 1,0,0) 


(d) P o (0,0,0);n=(fl,A,c) 


13. 


Determine whether the planes are parallel. 


(a) (-1,2,4) - (^-5,7 + 3 ,z-7) = 0; (2, -4, -8) - {x + 3,y + 5,z- 9) = 0 


(b) (3,0, -1) -(*+ \,y-2,z-3) = 0; (-1,0,3) - (x + \,y-z,z-3) = 0 


14 . 


Determine whether the planes are perpendicular. 


(a) (-2,1,4) -(x-\,y,z + 3) = 0-,(\, -2,1) - (* + 3. j; - 5,z) = 0 


(b) (3,0, -2) ■ {x A,y —1 ,z + 1)=0; (1, 1, 1) -(x,y,z) = 0 

Find the vector form of the equation of the line through p n and parallel to v. 

15. 

(a) P 0 ( — 1, 2, 3); v= (7, -1,5) 

(b) P 0 (2,0, - 1); v= (1, 1, 1) 

(c) P 0 (2, -4, 1); v= (0, 0, -2) 

(d) P o (0,0,0);v=(fl,A,c) 

Show that the line 

x = 0, y = t, z = t ( — oo <. t < 4 oo ) 

(a) lies in the plane 6x A- Ay — 4z = 0 

(b) is parallel to and below the plane 5x — 3y + 3z= 1 

(c) is parallel to and above the plane Sx \-2y — 2z = 3 

Find an equation for the plane through ( — 2, 1,7) that is perpendicular to the line x — 4 = 2t, y 4 2 = 3 t,z= — 5i- 

17. 

Find an equation of 

18. 

(a) the xy-plane 

(b) the x^-plane 

(c) the yz- plane 

Find an equation of the plane that contains the point (xq, z o) an ^ is 


(a) parallel to the z.y-plane 



(b) parallel to the yz-plane 


(c) parallel to the ^ z -plane 

Find an equation for the plane that passes through the origin and is parallel to the plane lx \ 4y — 2z + 3 = 0- 

20 . 

Find an equation for the plane that passes through the point ( 3 ; _ 6 , 7 ) and is parallel to the plane 5x — 2y l- z — 5 = 0- 

21 . 

Find the point of intersection of the line 
22 ‘ x — 9 = — 5t, y A- 1 = — t, z — 3 = t ( — oo < £ < + oo ) 

and the plane 2x - 3y A- 4z -I- 7 = 0- 

Find an equation for the plane that contains the line x=—\-\-3t, y = 5-\ 2t,z = 2 — t and is perpendicular to the plane 
23 " 2x — 4y I 2 z = 9- 

Find an equation for the plane that passes through ( 2 , 4 r — 1 ) and contains the line of intersection of the planes 
24* x — y — Az = 2 and — 2x A y 2z= 3- 

Show that the points ( — 1 , — 2, — 3), ( — 2, 0, 1)> ( — 4, — 1, — 1 ), and ( 2 , 0, 1) lie in the same plane. 

25. 

Find parametric equations for the line through ( — 2, 5, 0) that is parallel to the planes 2x f y — 4z = 0 and 
26 - - x 4 2y f 37 + 1 = 0 - 

Find an equation for the plane through ( — 2, 1, 5) that is perpendicular to the planes Ax — 2y + 2z = — 1 and 
27 " 3x + 3y-6z = 5- 

Find an equation for the plane through ( 2 , — 1 , 4 ) that is perpendicular to the line of intersection of the planes 
28" 4x 4 2y \- 2z = — 1 and 3x A- 6y A- 3z = 7- 

Find an equation for the plane that is perpendicular to the plane Bx — 2y A 6z= 1 and passes through the points 
29 ‘ Pi(-l,2,5)andp 2 (2, 1,4). 

Show that the lines 

x = 3 — 2t, y = 4 \-t, z = l — t ( — oo<£< + oo) 

and 

x = 5 A-21, y = 1 z= 1 A-t (— oo<^< + oo) 

are parallel, and find an equation for the plane they determine. 

Find an equation for the plane that contains the point (1 — 1 2) and the line x=Uy=£+ hz = —3 A- 2i- 

31. 

Find an equation for the plane that contains the line x = 1 \-1, y = 3t, z = 2t and is parallel to the line of intersection of the 
32 - planes _ x \ 2y fz = 0 and * 4 . z A- 1 = 0- 


Find an equation for the plane, each of whose points is equidistant from ( — 1, — 4, — 2) and (0, — 2, 2)- 

33 . 


Show that the line 

x — 5 = — t, y + 3 = 2t, 7 4 1 = — 5t ( — do <. £ < 4 oo ) 

is parallel to the plane — 2x A y + z — 9 = 0- 

Show that the lines 

x — 2— At, y — A = t, z—l = 0 (— oo < £ < 4 oo ) 

and 

x4l = 12i, y— l = 6t, z — 5 = 3t (—oo<t< + oo) 
intersect, and find the point of intersection. 

Find an equation for the plane containing the lines in Exercise 35. 

36 . 


Find parametric equations for the line of intersection of the planes 

37 . 

(a) — 3x 4 2y F z = — 5 and lx + 3y — 2z= —2 

(b) 5x — ly + 2z = 0 and y = 0 


Show that the plane whose intercepts with the coordinate axes are x = a, y = b, and z = c has equation 

—+y~ = i 
a b c 

provided that a, b, and c are nonzero. 

Find the distance between the point and the plane. 

(a) (3, 1, — 2); x 4 2^-2z = 4 

(b) (-1,2, 1); 2x4 37-47=1 

(c) (0,3, -2)\x-y-z = 3 


Find the distance between the given parallel planes. 

40 . 

(a) 3x — 4y 4 7 = 1 and 6x — 4 2z = 3 

(b) — 4x 4 y — 3z = 0 and 8x — 2^ 4 67 = 0 


(c) 2x — y 4 7 = 1 and 2x — y -t-z= — 1 


41 . 


Find the distance between the line x = 3t — y = 2 — t,z = t and each of the following points. 


(a) (0,0,0) 

(b) (2, 0, - 5) 

(c) (2,1,1) 


Show that if a, b, and c are nonzero, then the line 

42 

' x = *0 4- at, y = .yg + z = zo 4- ct ( — do<£<-|-og) 

consists of all points (x,y,z) that satisfy 

x -*n _ y -70 _ z-zn 
a b c 

These are called symmetric equations for the line. 

Find symmetric equations for the lines in parts (a) and (b) of Exercise 9. 

43 . 

Note See Exercise 42 for terminology. 

In each part, find equations for two planes whose intersection is the given line. 

44 . 

(a) x = 7 — 4U y = — 5 — 2t, z = 5 \-t ( — oo < £ < ■ I do) 


(b) x — At, y = 2U z = lt (— oo <t< -I do) 


Hint Each equality in the symmetric equations of a line represents a plane containing the line. See Exercise 42 for 
terminology. 


Two intersecting planes in 3-space determine two angles of intersection: an acute angle (0 < 8 < 90°) and its supplement 
180° — 9 (see the accompanying figure). If lq and n? are nonzero normals to the planes, then the angle between nj and 113 
9 or 180° — 9, depending on the directions of the normals (see the accompanying figure). In each part, find the acute angle 
of intersection of the planes to the nearest degree. 


(a) x = 0 and 2x — y + z — 4 = 0 

(b) x T 2_y — 2z = 5 and 6x — 3y \- 2z = 8 






Figure Ex-45 


Note A calculator is needed. 

Find the acute angle between the plane x —y — 3z = 5 and the line x = 2 — t, y = 2t, z = 3t — 1 to the nearest degree. 

46. 

Hint See Exercise 45. 


Discussion 

Discovery 


What do the lines r — rn | t w and r — rn _ t\ have in common? Explain. 

47. 


What is the relationship between the line x = xg i at> y =y$ + bt»z=ZQ l eft and the plane 
ax -f by 4- cz = 0? Explain your reasoning. 

Let r\ and r 2 be vectors from the origin to the points P\{x\,y\,z\) and ^ 2 (^ 2 .72. z 2% 
respectively. What does the equation 

r = (1 — i)i’i -I ti '2 (0<*<1) 
represent geometrically? Explain your reasoning. 

Write parametric equations for two perpendicular lines through the point (j 0 vn zn)- 

50. " 


How can you tell whether the line x | t w in 3-space is parallel to the plane 
51 * x = xg 4 H V 1 h^2 v 2 ? 

Indicate whether the statement is true (T) or false (F). Justify your answer. 

52. 


(a) If a, b, and c are not all zero, then the line x = at, y = bt, z = ct is perpendicular to the 
plane ax 4 by \- cz = Gl¬ 
ib) Two nonparallel lines in 3-space must intersect in at least one point. 

(c) If u, v, and w are vectors in 3-space such that u -| v + w = 0, then the three vectors lie in 
some plane. 

(d) The equation x = tx represents a line for every vector v in 2-space. 
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Abbreviations 


GPS Global Positioning System 
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Chapter 3 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Section 3.1 

Tl. (Vectors) Read your documentation on how to enter vectors and how to add, subtract, and multiply them by scalars. Then 
perform the computations in Example 1. 


T2. (DrawingVectors) If you are using a technology utility that can draw line segments in two or three-dimensional space, try 
drawing some line segments with initial and terminal points of your choice. You may also want to see if your utility allows 
you to create arrowheads, in which case you can make your line segments look like geometric vectors. 


Section 3.3 

Tl. (Dot Product and Norm) Some technology utilities provide commands for calculating dot products and norms, whereas 
others provide only a command for the dot product. In the latter case, norms can be computed from the formula ||v|| = 
Read your documentation on how to find dot products (and norms, if available), and then perform the computations in 
Example 2. 


T2. (Projections) See if you can program your utility to calculate proj au when the user enters the vectors a and u. Check your 
work by having your program perform the computations in Example 6. 

Section 3.4 

Tl. (Cross Product) Read your documentation on how to find cross products, and then perform the computation in Example 1. 


T2. (Cross Product Formula) If you are working with a CAS, use it to confirm Formula 1. 


T3. (Cross Product Properties) If you are working with a CAS, use it to prove the results in Theorem 3.4.1. 




T4. (Area of a Triangle) See if you can program your technology utility to find the area of the triangle in 3-space determined by 
three points when the user enters their coordinates. Check your work by calculating the area of the triangle in Example 4. 


T5. (Triple Scalar Product Formula) If you are working with a CAS, use it to prove Formula 7 by showing that the difference 
between the two sides is zero. 


T6. (Volume of a Parallelepiped) See if you can program your technology utility to find the volume of the parallelepiped in 
3-space determined by vectors u, v, and w when the user enters the vectors. Check your work by solving Exercise 10 in 
Exercise Set 3.4 
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4 

CHAPTER 


Euclidean Vector Spaces 


NTRODUCTION The idea of using pairs of numbers to locate points in the plane and triples of numbers to locate points 
in 3-space was first clearly spelled out in the mid-seventeenth century. By the latter part of the eighteenth century, 
mathematicians and physicists began to realize that there was no need to stop with triples. It was recognized that quadruples 
of numbers ( fll> a 2 , a 3 , a 4 ) could be regarded as points in “four-dimensional” space, quintuples ( flli fl2i « 3 , & 4 , aj) as points 
in “five-dimensional” space, and so on, an n-tuple of numbers being a point in “ n-dimensional” space. Our goal in this chapter 
is to study the properties of operations on vectors in this kind of space. 
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4.1 

EUCLIDEAN /7-SPACE 

Although our geometric visualization does not extend beyond 3-space, it is 
nevertheless possible to extend many familiar ideas beyond 3-space by 
working with analytic or numerical properties of points and vectors rather than 
the geometric properties. In this section we shall make these ideas more 
precise. 

Vectors in /7-Space 


We begin with a definition. 


DEFINITION 


If n is a positive integer, then an ordered n-tuple is a sequence of n real numbers {a\, ai,a n )- The set of all ordered 
n-tuples is called n-space and is denoted by R n . 


When n = 2 or 3, it is customary to use the terms ordered pair and ordered triple, respectively, rather than ordered 2-tuple and 
ordered 3-tuple. When n = \, each ordered n-tuple consists of one real number, so p} may be viewed as the set of real numbers. 
It is usual to write R rather than p} for this set. 


It might have occurred to you in the study of 3-space that the symbol ( fll> aj, cti) has two different geometric interpretations: it 
can be interpreted as a point, in which case a i, 0 , 2 , and are the coordinates (Figure 4.1.1a), or it can be interpreted as a vector, 
in which case a 1 , aj, and are the components (Figure 4.1.1b). It follows, therefore, that an ordered n-tuple {a\, aj..., a H ) can 
be viewed either as a “generalized point” or as a “generalized vector”—the distinction is mathematically unimportant. Thus we 
can describe the 5-tuple (-2, 4, 0, 1, 6 ) either as a point in p$ or as a vector in py. 



(«) 


Figure 4.1.1 



The ordered triple ( fll> £ 3 ) can be interpreted geometrically as a point or as a vector. 












DEFINITION 


Two vectors u = ( u 1 , u 2 ,..., u„) and v = ( Vl , V 2 ,..., v„) ini?" are called equal if 


“1=V1. U 2 = v 2 ,..., U n = v n 

The sum u 4 - y is defined by 

u +v= (u 1 f v\,U 2 + V 2 , u M + v„) 

and if k is any scalar, the scalar multiple jfcu is defined by 


hi = (ku i ? ku2, hiy{) 


The operations of addition and scalar multiplication in this definition are called the standard operations on pf. 


The zero vector in R n is denoted by 0 and is defined to be the vector 

0 = ( 0 , 0 ,..., 0 ) 

If u = (tt j ; u'i ,..., u„) is any vector in R n , then the negative (or additive inverse ) of u is denoted by _ u and is defined by 

-u= (-ui, -u n ) 


The difference of vectors in R n is defined by 


v — u = v 4 ( — u) 


or, in terms of components, 


v-u = (vi -ui, V2-U2,—, 


Some Examples of Vectors in Higher-Dimensional Spaces 


Experimental Data A scientist performs an experiment and makes n numerical measurements each time the 
experiment is performed. The result of each experiment can be regarded as a vector y = (y\,y2, y n ) ' n R n i n which 
y i, y% are the measured values. 


Storage and Warehousing A national trucking company has 15 depots for storing and servicing its trucks. At each 
point in time the distribution of trucks in the service depots can be described by a 15-tuple x = (jq, x 2 ,x\f) in which 
jq is the number of trucks in the first depot, x 2 is the number in the second depot, and so forth. 


Electrical Circuits A certain kind of processing chip is designed to receive four input voltages and produces three 
output voltages in response. The input voltages can be regarded as vectors in pf and the output voltages as vectors in g} 
. Thus, the chip can be viewed as a device that transforms each input vector v = ( Vl , V 2 , V 3 , V 4 ) in j ? 4 into some output 
vector y- = (w 1 , >^ 2 , w 3 ) in g?. 


Graphical Images One way in which color images are created on computer screens is by assigning each pixel (an 
addressable point on the screen) three numbers that describe the hue, saturation, and brightness of the pixel. Thus, a 
complete color image can be viewed as a set of 5-tuples of the form v = y, h, s, b) in which x and y are the screen 
coordinates of a pixel and h, s, and b are its hue, saturation, and brightness. 







Economics Our approach to economic analysis is to divide an economy into sectors (manufacturing, services, utilities, 
and so forth) and to measure the output of each sector by a dollar value. Thus, in an economy with 10 sectors the 
economic output of the entire economy can be represented by a 10-tuple s = ( Sl> sio) * n which the numbers sj, sj, 
..., sjq are the outputs of the individual sectors. 


Mechanical Systems Suppose that six particles move along the same coordinate line so that at time t their coordinates 
are xj, ^ 2 , ..., x$ and their velocities are vj, V 2 , •••, vg, respectively. This information can be represented by the vector 

v = (*i, *2, * 3 , * 4 , x$, *6, vi, V2, V3, V4, V5, vg, t) 

in $}?>. This vector is called the state of the particle system at time t. 

Physics In string theory the smallest, indivisible components of the Universe are not particles but loops that behave 
like vibrating strings. Whereas Einstein's space-time universe was four-dimensional, strings reside in an 11-dimensional 
world. 


Properties of Vector Operations in r?-Space 

The most important arithmetic properties of addition and scalar multiplication of vectors in pf are listed in the following 
theorem. The proofs are all easy and are left as exercises. 


THEOREM 4.1.1 

Properties of Vectors in R n 

Ifn = (u\,U 2 , Un)’ v = (vi, V 2 , - v H )> and w = (wj, m? 2 , ..., w H ) are vectors in R n and k and m are scalars, then: 

(a) u T v = v T u 

(b) u + (v + w) = (u -I- v) -I- w 

(c) u + 0 = 0 I u = u 

(d) u + ( — u) = 0» that is, u — u = 0 

(e) k{m\\) = (km)\\ 

(f) jt(u -I- v) = -I- kv 

(g) (£H »*)u = jfcii +wu 




(h) lu = u 


This theorem enables us to manipulate vectors in R n without expressing the vectors in terms of components. For example, to 
solve the vector equation x | u — v for x, we can add — u to both sides and proceed as follows: 

(x + u) 4 ( - u) = v + ( - u) 
x 4- (u — u) = v — u 
X + 0 = v - u 

X = v — u 

The reader will find it instructive to name the parts of Theorem 4.1.1 that justify the last three steps in this computation. 

Euclidean A7-Space 

To extend the notions of distance, norm, and angle to R n , we begin with the following generalization of the dot product on R 2 
and R 3 [Formulas 3 and 4 of Section 3.3], 


DEFINITION 


If u= (ui,«2, u n) and v= (vi, V 2 , 

v„) are any vectors in R n , then the Euclidean inner product u . v is defined by 

u-v = uivi 4 U 2 V 2 4 ... 4 U„V H 


Observe that when n = 2 or 3, the Euclidean inner product is the ordinary dot product. 


Inner Product of Vectors in R A 


The Euclidean inner product of the vectors 


in r^ is 


u= (-1,3, 5, 7) and v=(5, -4,7,0) 
u-v= (- 1)(5) 4 - (3)( — 4) + (5)(7) f (7)(0) = 18 


Since so many of the familiar ideas from 2-space and 3-space carry over to n-space, it is common to refer to R n , with the 
operations of addition, scalar multiplication, and the Euclidean inner product, as Euclidean n-space. 

The four main arithmetic properties of the Euclidean inner product are listed in the next theorem. 


♦ 


THEOREM 4.1.2 


Properties of Euclidean Inner Product 

Ifu, V, and w are vectors in R n and k is any scalar , then: 











( a ) U - V = V - u 

(b) (u -|- v) - w = u - w 4- v - w 

(c) (,hl) - V = jfc(u - v) 

(d) v - v > 0. Further, v ■ v = 0 if and only ify = 0- 


We shall prove parts ( b ) and (d) and leave proofs of the rest as exercises. 


Application of Dot Products to ISBNs 

Most books published in the last 25 years have been assigned a unique 10-digit number called an International Standard 
Book Number or ISBN. The first nine digits of this number are split into three groups—the first group representing the 
country or group of countries in which the book originates, the second identifying the publisher, and the third assigned to the 
book title itself. The tenth and final digit, called a check digit, is computed from the first nine digits and is used to ensure that 
an electronic transmission of the ISBN, say over the Internet, occurs without error. 

To explain how this is done, regard the first nine digits of the ISBN as a vector b in pf, and let a be the vector 

a = (1, 2, 3,4, 5,6,7, 8,9) 

Then the check digit c is computed using the following procedure: 


Form the dot product a I,. 

Divide - b by 11, thereby producing a remainder c that is an integer between 0 and 10, inclusive. The check digit is 
taken to be c, with the proviso that c = 10 is written as X to avoid double digits. 

For example, the ISBN of the brief edition of Calculus, sixth edition, by Howard Anton is 

0-471 - 15307-9 

which has a check digit of 9. This is consistent with the first nine digits of the ISBN, since 

a -b = (1, 2, 3, 4, 5, 6 , 7, 8 , 9) ■ (0, 4, 7, 1, 1, 5, 3, 0, 7) = 152 

Dividing 152 by 11 produces a quotient of 13 and a remainder of 9, so the check digit is c = 9- If an electronic order is placed 
for a book with a certain ISBN, then the warehouse can use the above procedure to verify that the check digit is consistent 
with the first nine digits, thereby reducing the possibility of a costly shipping error. 


Proof (b) Let u = (hj, u n ), v = (vi, V 2 , v H )> and w = (uq, W 2 ,w H )- Then 

(u-| v) w= Oi -I- VI, U2 4 - V2, U n -I- v H ) - Oi, W2, w H ) 

= (“l+vi)wi + (u2 + V2)w 2 4 ----4 - (u h +v„)w h 
= (iqwi 4 - U 2 V /2 + --- + + (vpvi 4 - V2W2 4 -... 4 - v H w„) 

= u - w 4 - v - w 




We have v _ v _ v 2 f > 0- Further, equality holds if and only if v j = V 2 = — = v H = 0—that is, if and 

only if v = 0 - 


Length and Distance in 


Theorem 4.1.2 allows us to perform computations with Euclidean inner products in much the same way as we perform them 
with ordinary arithmetic products. For example, 

(3u 4- 2v) - (4u 4 v) = (3u) - (4u 4 v) 4 (2v) - (4u 4 v) 

= (3u) - (4u) 4 (3u) ■ v 4 (2v) - (4u) 4 (2v) ■ v 
= 12(u -u) I- 3(u- v) 4 8 (v ■ u) 4 2(v ■ v) 

= 12 (u - u) 4 11 (u - v) 4 2 (v - v) 


The reader should determine which parts of Theorem 4.1.2 were used in each step. 


♦ 


Norm and Distance in Euclidean n-Space 

By analogy with the familiar formulas in R 2 and R- : , we define the Euclidean norm (or Euclidean length ) of a vector 
u= («i,U 2 ,-~, u H ) in 7?” by 

||u|| = (u-u) 1/2 = /uj 4^2 4-- + aJ (1) 

[Compare this formula to Formulas 1 and 2 in Section 3.2.] 

Similarly, the Euclidean distance between the points u = (u\, U 2 , u H ) and y = (vj, V 2 ,v H ) in R n is defined by 

d (u, v) = ||u - v|| = ^(ai -vi) 2 4 (u 2 - v 2 ) 2 4-4 (a H -v H ) 2 (2) 

[See Formulas 3 and 4 of Section 3.2.] 


EXAMPLE 3 Finding Norm and Distance 

If u = (1, 3, — 2, 7) and v = (0, 7, 2, 2), then in the Euclidean space R 4 , 

INI = l/(l) 2 4 (3 ) 2 4 ( - 2) 2 4 (7 ) 2 = /63 = 3^7 
and 

d(n, v) = /(I-0) 2 4 (3-7) 2 4 ( - 2 - 2 ) 2 4 (7 - 2 ) 2 = ^58 

♦ 

The following theorem provides one of the most important inequalities in linear algebra: the Cauchy-Schwarz inequality. 

THEOREM 4.1.3 

Cauchy-Schwarz Inequality in R n 











If n= (hi, u 2 , u n ) and v = (vi, V2,v H ) are vectors in R n , then 


|u-v|< 


U 


( 3 ) 


In terms of components, 3 is the same as 


|«ivi 4- u 2 V2 + ■ 


■ -|- 3 “ u 2 3 ""*" 4 ^ h ) 


1/2 


(v? 3 v 2 2 


I —f v h) 


1/2 


(4) 


We omit the proof at this time, since a more general version of this theorem will be proved later in the text. However, for 
vectors in p} and p}, this result is a simple consequence of Formula 1 of Section 3.3: If u and v are nonzero vectors in pf or pf, 
then 


|u - v| = |||u|| ||v|| cos 0\ = ||u|| ||v|| |cos 9\ < ||u|| ||v|| 

and if either u = 0 or v = 0, then both sides of 3 are zero, so the inequality holds in this case as well. 
The next two theorems list the basic properties of length and distance in Euclidean n-space. 



Augustin Louis (Baron de) Cauchy 


Augustin Louis (Baron de) Cauchy (1789-1857), French mathematician. Cauchy's early education was acquired from his 
father, a barrister and master of the classics. Cauchy entered L'Ecole Polytechnique in 1805 to study engineering, but because 
of poor health, he was advised to concentrate on mathematics. His major mathematical work began in 1811 with a series of 
brilliant solutions to some difficult outstanding problems. 

Cauchy's mathematical contributions for the next 35 years were brilliant and staggering in quantity: over 700 papers filling 
26 modem volumes. Cauchy's work initiated the era of modern analysis; he brought to mathematics standards of precision 
and rigor undreamed of by earlier mathematicians. 

Cauchy's life was inextricably tied to the political upheavals of the time. A strong partisan of the Bourbons, he left his wife 
and children in 1830 to follow the Bourbon king Charles X into exile. For his loyalty he was made a baron by the ex-king. 
Cauchy eventually returned to France but refused to accept a university position until the government waived its requirement 
that he take a loyalty oath. 





It is difficult to get a clear picture of the man. Devoutly Catholic, he sponsored charitable work for unwed mothers and 
criminals and relief for Ireland. Yet other aspects of his life cast him in an unfavorable light. The Norwegian mathematician 
Abel described him as “mad, infinitely Catholic, and bigoted.” Some writers praise his teaching, yet others say he rambled 
incoherently and, according to a report of the day, he once devoted an entire lecture to extracting the square root of seventeen 
to ten decimal places by a method well known to his students. In any event, Cauchy is undeniably one of the greatest minds 
in the history of science. 



Herman Amandus Schwarz 


Herman Amandus Schwarz (1843-1921), German mathematician. Schwarz was the leading mathematician in Berlin in the 
first part of the twentieth century. Because of a devotion to his teaching duties at the University of Berlin and a propensity for 
treating both important and trivial facts with equal thoroughness, he did not publish in great volume. He tended to focus on 
narrow concrete problems, but his techniques were often extremely clever and influenced the work of other mathematicians. 
A version of the inequality that bears his name appeared in a paper about surfaces of minimal area published in 1885. 


THEOREM 4.1.4 


Properties of Length in r" 

If u and v are vectors in pf and k is any scalar, then: 

(a) Nl>0 

(b) ||u|| = 0 if and only ifu = 0 

(c) ll*u|| = 1*1 INI 

(d) ||u -|- v|| < ||u|| 4- 11v11 (Triangle inequality) 


We shall prove (c) and (d) and leave (a) and (b) as exercises. 




)of (c) If u = (uj, U2, u n ), then jbi= (Aaj, ku2, ku n ), so 


IM = l/(^ui) 2 + (fe/2) 2 + -+ C^h) 2 
= |*| |/uJ + ajH- |-k 2 

= |*|ll«ll 


Proof (d) 


\\n + v || 2 = (u 4 v) - (u + v) = (u ■ u) 4- 2 (u - v) + (v - v) 

= ||u|| 2 + 2(u-v) + ||v|| 2 

< ||u || 2 + 2 |u ■ v| |- ||v || 2 

< ||u || 2 f 2 ||u||||v|| 4 ||v|| 2 Cauchy—Schwarz inequality 

= (IHI + ||v ||) 2 

The result now follows on taking square roots of both sides. 

i 

Part (c) of this theorem states that multiplying a vector by a scalar k multiplies the length of that vector by a factor of |.t| (Figure 
4.1.2a). Part (d) of this theorem is known as the triangle inequality because it generalizes the familiar result from Euclidean 
geometry that states that the sum of the lengths of any two sides of a triangle is at least as large as the length of the third side 
(Figure 4.1.2b). 



(a) IHuJI = |*|Hull 



ih) || u + v || < || u || + || v 

Figure 4.1.2 


The results in the next theorem are immediate consequences of those in Theorem 4.1.4, as applied to the distance function 
^(u, v) on R n . They generalize the familiar results for R 2 and RR. 


THEOREM 4.1.5 




Properties of Distance in R n 

Ifu, v, and w are vectors in R n and k is any scalar, then: 

(a) d (11, v) > 0 

(b) d (u, v) = 0 if and only if u = v 

(c) d (u, v) = d (v, u) 

(d) d (u, v) < d (v, w) 4- d (w, v) (Triangle inequality) 

We shall prove part (d) and leave the remaining parts as exercises. 

Proof (d) From 2 and part ( d) of Theorem 4.1.4, we have 

d (u, v) = ||u - v|| = || (u - w) + (w - v) || 

< ||u — w|| 4- ||w — v|| = d (u, w) H d (w, v) 

■ 

Part (d) of this theorem, which is also called the triangle inequality, generalizes the familiar result from Euclidean geometry that 
states that the shortest distance between two points is along a straight line (Figure 4.1.3). 



u 


r/(u, w) < rfdi. v) + iliv, w) 
Figure 4.1.3 


Formula 1 expresses the norm of a vector in terms of a dot product. The following useful theorem expresses the dot product in 
terms of norms. 


THEOREM 4.1.6 


Ifu and v are vectors in R n with the Euclidean inner product, then 




Proof 


u-v = I||ti + v|| 2 -I||u-v| 


(6) 


||u 4 v|| 2 = (u 4 v) - (u 4 v) = ||u|| 2 4- 2(u - v) 4 ||v|| 2 
||u — v|| 2 = (u — v) - (u — v) = ||u|| 2 — 2(u- v) -I- ||v|| 2 

from which 6 follows by simple algebra. 

Some problems that use this theorem are given in the exercises. 

Orthogonality 

Recall that in the Euclidean spaces p} and p}\ two vectors u and v are defined to be orthogonal (perpendicular) if u . y = 0 
(Section 3.3). Motivated by this, we make the following definition. 


DEFINITION 


Two vectors u and v in R n are called orthogonal if u - v = 0- 


EXAMF Orthogonal Vectors in R 4 


In the Euclidean space p 4 the vectors 


are orthogonal, since 


u= (-2, 3, 1.4) and v= (1,2,0, -1) 
u v = ( -2)(1) + (3)(2) 4 (1)(0) 4 (4)(- 1) = 0 


♦ 


Properties of orthogonal vectors will be discussed in more detail later in the text, but we note at this point that many of the 
familiar properties of orthogonal vectors in the Euclidean spaces p 2 and p} continue to hold in the Euclidean space P n . For 
example, if u and v are orthogonal vectors in p} or p}, then «, v, and u 4. v form the sides of a right triangle (Figure 4.1.4); thus, 
by the Theorem of Pythagoras, 

I|u 4 v|| 2 = ||u|| 2 4 ||v|| 2 

The following theorem shows that this result extends to R n . 










Figure 4.1.4 


THEOREM 4.1.7 

Pythagorean Theorem in r" 

If u and v are orthogonal vectors in R n with the Euclidean inner product, then 

||u + v || 2 = ||u|| 2 +IM | 2 


Proof 


u 4 v|| 2 = (u h v) - (u 4 v) = ||u|| 2 + 2(u - v) 4- ||v|| 2 = ||u|| 2 



Alternative Notations for Vectors in R n 


It is often useful to write a vector u = (ui, u- 2 , ..., h h ) in R n in matrix notation as a row matrix or a column matrix: 

or u = [hi U 2 ... h h ] 


u = 


U 1 
u 2 


Uy 


This is justified because the matrix operations 



'hi ' 


'vi ' 


'hi ■ 

1 VI ' 


'hi ' 


ku 1 

U + V = 

u 2 

4 

V2 

= 


1 V2 

■--i- 

II 

h 

u 2 

= 

^2 


H h 


v H 


H h 

1 v H 


H h 


ku n 


or 

U + Y =[«1 h 2 ... u„] 4 [vi v 2 ... v„] 

= I«i +vi U 2 + v 2 ... + v„] 

iu = i[Hi U2 ... h m ] = [&hi ku2 --- kuyi\ 
produce the same results as the vector operations 

u+ v= (u i, u 2 , h„) 4 (vi, V 2 , v M ) = (hi 4 vi, H2 + v 2 ,..., h h 4 v H ) 
iu = k(u\, U 2 , h h ) = (jfcui, hi 2 , kuyf} 

The only difference is the form in which the vectors are written. 


A Matrix Formula for the Dot Product 

















If we use column matrix notation for the vectors 



~Ul ' 


"vi ' 

u = 

u 2 

g 

<1 

II 

v 2 


Uyi 




and omit the brackets on ] x 1 matrices, then it follows that 


v ii= [vi v 2 


V H ] 


“ 1 
u 2 

W VI 


[« 1 V 1 4- u 2 V 2 + “ + a«v„] = [u- v] =u - v 


Thus, for vectors in column matrix notation, we have the following formula for the Euclidean inner product: 


ll-V = V 11 


(7) 


For example, if 


then 


’ - r 


5" 

3 

and v = 

-4 

5 


7 

7 


0 


u-v = v r u=[5 -4 7 0] 



[18] = 18 


If A is an n x « matrix, then it follows from Formula 7 and properties of the transpose that 

j4u - v = v ^ (j4u) = (v^)u = ( A T v) u = u - A 
u ■ j4v = (j4v) = (v 7 A T )n = v ^(A ^u) = A ■ v 

The resulting formulas 


An - v = u - A 


( 8 ) 


u- Av = A T n- v 

provide an important link between multiplication by an n x n matrix A and multiplication by ^ 7. 


(9) 


EXAMPLE Verifying That Auv = u-A r v 


Suppose that 


m 

■■J 

1 

i_ 


' = r 


i— 

■■j 

i 

i_ 

2 4 1 

, U = 

2 

, v = 

0 

-1 0 1 


4 


-1 

_i 


Then 






















An = 


1-2 3' 

' -l" 


' 7' 

2 4 1 

2 

= 

10 

-1 0 1 

4 


5 

12 -1' 

' -2" 


' -7 

-2 4 0 

0 

= 

4 

3 1 1 

5 


- 1 


from which we obtain 

j4u- v = 7( — 2) 4 10(0)+ 5(5) = 11 
xl-A t y = (- l)(-7) + 2(4) +4(- 1) = 11 

Thus . y = u - A T v as guaranteed by Formula 8. We leave it for the reader to verify that 9 also holds. 


A Dot Product View of Matrix Multiplication 


Dot products provide another way of thinking about matrix multiplication. Recall that if A = [a,j ] is an m x r matrix and 
B = is an r x n matrix, then the i yth entry of AB is 

a i\b 1/ F a i2&2j 4 V <% 


which is the dot product of the /th row vector of A 
and the yth column vector of B 


[fl;l 2 ... iSj'j-] 


b 2 j 


Thus, if the row vectors of A are rj, i’ 2 , 
expressed as 


r m and the column vectors of B are cj, 02, ..., c„, then the matrix product AB can be 



i’l ■ 

ci 

I’l ■ 

c 2 - 

- ri ■ 

■ c H 

AB = 

r 2 ' 

■ ci 

r 2 ■ 

c 2 - 

1’2 ■ 

' 



■Cl 

l’m 

■C2 - 

I’m 



( 10 ) 


In particular, a linear system j4x = b can be expressed in dot product form as 


' i’i ■ *" 


V 

1’2 * 


h 

r m ' 1 


b m 


where rj, i’2, ..., r m are the row vectors of A, and b\^b 2 ^ •••> are the entries of b. 


(ID 


EXAMPLE 6 A Linear System Written in Dot Product Form 


The following is an example of a linear system expressed in dot product form 11. 
























Dot Product Form 


System 


3*i — 4*2 +*3 = 1 

(3, -4, 1) ■ {x\,x 2 , x 3 ) 


V 

2*1 — 7*2 — 4*3 = 5 

(2, -7, -4) - (xi,X 2 ,x 3 ) 

= 

5 

*1 + 5*2 — 8*3 = 0 

(1, 5, - 8 ) ■ {x\,x 2 , xj) 


0 


Exercise Set 4.1 



Click here for Just Ask! 


Let u = ( — 3, 2, 1, 0), v= (4, 7, - 3, 2), and w = (5, -2,8, l).Find 

(a) v-w 

(b) 2u + 7v 


(c) - U + (v - 4w) 

(d) 6 (u — 3v) 

(e) -v-w 

(f) ( 6 v — w) — (4u + v) 


Let u, v, and w be the vectors in Exercise 1. Find the vector x that satisfies 5 X — 2v = 2(w — 5x)- 


Let uj = ( - 1, 3, 2, 0), n 2 = (2, 0,4, - l),u 3 = (7, 1, 1, 4), and U4 = ( 6 , 3, 1, 2)- Find scalars c\,c 2 ,c 3 , and c 4 such that 

3 

• c im 4- c 2 \\ 2 + + C 4 U 4 = (0> 5, 6, - 3)- 

Show that there do not exist scalars c\,c 2 , and c 3 such that 

4 ciCl.O, 1,0)+<72(1.0. —2, 1) + c 3 (2, 0, 1, 2) = (1, -2,2,3) 

In each part, compute the Euclidean norm of the vector. 

5 . 


(a) (-2,5) 


(b) ( 1 , 2 ,- 2 ) 









(c) (3, 4, 0,-12) 


(d) (-2, 1,1,-3, 4) 


Let u = (4, 1, 2, 3), y = (0, 3, 8, —2), and w = (3, 1, 2, 2)- Evaluate each expression. 

(a) II« + t|| 

(b) IN + ||v|| 

(C ) || — 2u|| 4- 2||u|| 

(d) II3u — 5v 4- w|| 


7 - 

8 . 


9. 


Show that if v is a nonzero vector in R n , then (1 / ||v||)v has Euclidean norm 1. 


Let v = ( — 2, 3, 0, 6)- Find all scalars k such that ||£v|| = 5. 


Find the Euclidean inner product u - y. 


(a) u=(2,5),v=(-4,3) 


(b) u= (2, 8, 2), v= (0, 1.3) 


(c) u= (3, 1,4, -5),v=(2, 2, -4,-3) 

(d) u = ( — 1, 1,0,4, -3),v=(-2, -2,0,2, - 1) 




10 . 

(a) Find two vectors in p} with Euclidean norm 1 whose Euclidean inner product with (3, -1) is zero. 

(b) Show that there are infinitely many vectors in p 3 with Euclidean norm 1 whose Euclidean inner product with (1, -3, 
5) is zero. 

Find the Euclidean distance between u and v. 

11 . 

(a) u= (1, -2),v=(2, 1) 

(b) u= (2, — 2, 2)» v = (0, 4, -2) 

(c) u= (0, -2, -1, l),v=(-3, 2,4,4) 

(d) u= (3, -3, -2,0, — 3). v = ( — 4, 1, -1,5,0) 

Verify parts (b), ( e ), (/), and (g) of Theorem 4.1.1 for u = (2, 0, - 3, 1), v= (4, 0, 3, 5), w = (1, 6, 2, - 1), k = 5, and 
12 ' / = -3- 

Verify parts ( b ) and (c) of Theorem 4.1.2 for the values of u, v, w, and k in Exercise 12. 

13. 

In each part, determine whether the given vectors are orthogonal. 

14. 

(a) u = ( — 1, 3, 2). v= (4, 2, -1) 

(b) u= (-2, -2, -2),v=(l, 1.1) 

(e) u= (hi, U 2 , v= (0, 0, 0) 

(d) u= (-4, 6, -10, 1), v = (2, 1, -2,9) 

(e) u= (0,3, -2, l),v=(5, 2, -1,0) 

(f) u= (a, b), v = ( — b, a) 


For which values of k are u and v orthogonal? 


(a) u=(2, 1, 3), v= (1, 7, £) 


(b) u = (k, k, 1), v= (k, 5, 6) 

Find two vectors of norm 1 that are orthogonal to the three vectors u=(2, 1, — 4, 0), v = ( — 1, —1,2,2), and 
16 ' w= (3, 2, 5,4). 

In each part, verify that the Cauchy-Schwarz inequality holds. 

17. 

(a) u= (3, 2), v= (4, -1) 

(b) u = ( — 3, 1, 0), v = (2, -1,3) 

(c) u= (-4, 2, 1), v= (8, -4, -2) 

(d) u= (0, -2,2, 1), v= (- 1, -1, 1, 1) 

In each part, verify that Formulas 8 and 9 hold. 


II 

'2 - r 

, u = 

■ 3 ' 

, V = 

'-2' 


3 4 


1 


6 


(b) 

' - 1 2 4' 


' - r 


o' 

A = 

3 1 0 

, u = 

2 

, V = 

2 


5-2 3 


5 


-4 


Solve the following linear system for xj, and * 3 . 

19. 

(1, -1,4)-(x 1 , I2 ^3) = 10 
(3, 2, 0) - (xi, *2> * 3 ) = 1 
(4, -5, -l).(x 1>l2 ,x3) =7 

Find u . v given that ||u 4 v|| = 1 and ||u — v|| = 5. 

20 . 

Use Theorem 4.1.6 to show that u and v are orthogonal vectors in R n if ||u 4 v|| = ||u — v||. Interpret this result 
21* geometrically in pp-. 





The formulas for the vector components in Theorem 3.3.3 hold in R M as well. Given that a = (—1, 1,2,3) and 
22* || = (2, 1,4, — 1), find the vector component of u along a and the vector component of u orthogonal to a. 

Determine whether the two lines 

23 - r=(3,2,3, -l)+;(4,6,4, -2) and r = (0, 3, 5, 4) 4 s(l. - 3, -4, - 2) 
intersect in j? 4 . 

Prove the following generalization of Theorem 4.1.7. If vj, v 2 , ..., v r are pairwise orthogonal vectors in R™, then 
24 " ||vi+v 2 + -+v>|| 2 =||vi|| 2 + ||v 2 || 2 H-h ||V ,|| 2 


25 . 


Prove: If u and v are n y..\ matrices and A is an n x « matrix, then 


ni ni ji, T" T" T" T" 

(V T A T Ax i) < (u T A T Ax 1) ( V T A T Av) 


TaT 


„T & T „ 


Use the Cauchy-Schwarz inequality to prove that for all real values of a, b, and 0, 

(a cos 9 A b sin#) 2 < a 2 + & 2 

Prove: If u, v, and w are vectors in R n and k is any scalar, then 
(a) u - (£v) = jt(n - v) 


(b) u - (v 4 w) = u - v 4 u - w 


Prove parts (a) through ( d) of Theorem 4.1.1. 

28 . 


Prove parts ( e ) through (h) of Theorem 4.1.1. 

29 . 

Prove parts ( a ) and (c) of Theorem 4.1.2. 

30 . 

Prove parts ( a ) and ( b ) of Theorem 4.1.4. 

31 . 

Prove parts (a), ( b ), and (c) of Theorem 4.1.5. 

32 . 

Suppose that a i, aj, a H are positive real numbers. In p}, the vectors yj = (a\, 0) and y 2 = (0, a-R) determine a rectan; 
33 * of area A = a\ai ( see th e accompanying figure), and in g 3 , the vectors Vl = {a\, 0, 0), v 2 = (0, a 2 , 0), and y 3 = (0, 0, 

determine a box of volume y = a\a 2 <-tz ( see the accompanying figure). The area A and the volume V are sometimes callec 
the Euclidean measure of the rectangle and box, respectively. 


(a) How would you define the Euclidean measure of the “box” in R n that is determined by the vectors 


vi = (a\, 0, 0,..., 0) v 2 = (0, a 2 , 0,..., 0),..., 


v H = (0, 0, 0,..., a H ) ? 



(b) How would you define the Euclidean length of the “diagonal” of the box in part (a)? 


lO- rM / 



iti h U) 


Area A = a \it 2 

Figure Ex-33 



Discussion 
Discovery 34 

(a) Suppose that u and v are vectors in R n . Show that 

||u+v|| 2 +||u-v|| 2 = 2(||ti|| 2 f ||v|| 2 ) 

(b) The result in part (a) states a theorem about parallelograms in p}. What is the theorem? 

35. 

(a) If u and v are orthogonal vectors in R n such that ||u|| = 1 and ||v|| = 1, then ^( u , v) = 

(b) Draw a picture to illustrate this result. 


In the accompanying figure the vectors u, v, and u _ v form a triangle in p}, and fj denotes the 
36* angle between u and v. It follows from the law of cosines in trigonometry that 

i2 


llu -v|| 2 = Hull 2 4- 


' — 2||u|| ||v||cos 9 

Do you think that this formula still holds if u and v are vectors in P n l Justify your answer. 



Figure Ex-36 


37 . 


Indicate whether each statement is always true or sometimes false. Justify your answer by giving 
a logical argument or a counterexample. 














(a) If || u _|_ v || 2 = || u || 2 4 . || v || 2 , then u and v are orthogonal. 

(b) If u is orthogonal to v and w, then u is orthogonal to v -|- w- 

(c) If u is orthogonal to v -|- w, then u is orthogonal to v and w. 

(d) If ||u — v|| = 0, then u = v. 

(e) If ||Aai|| =jt||u||, then^> 0. 


Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 



4.2 

LINEAR 

TRANSFORMATIONS 
FROM R n TO R m 


Functions from R n to R 

Recall that a function is a rule/that associates with each element in a set A one and only one element in a set B. If/associates 
the element b with the element a, then we write b = j (a) and say that b is the image of a under/or that f (a) is the value of/at 
a. The set A is called the domain of/and the set B is called the codomain off. The subset of B consisting of all possible values 
for/as a varies over A is called the range off. For the most common functions, A and B are sets of real numbers, in which case/ 
is called a real-valued function of a real variable. Other common functions occur when B is a set of real numbers and A is a set 
of vectors in pf, pf, or, more generally, R n . Some examples are shown in Table 1. Two functions / j and / 2 are regarded as 
equal, written / j = / 2 , if they have the same domain and / j (a) = f 2 (a) f° r a h a in the domain. 


Table 1 


Formula 

Example 

Classification 

Description 

/w 

/W =^ 2 

Real-valued function of a real variable 

Function from R to R 


f(x,y)=x 2 + y 2 

Real-valued function of two real 
variables 

Function from r} to 

R 

f(*,y,z) 

f(x,y,z)=x l + y 2 \z l 

Real-valued function of three real 
variables 

Function from r} to 

R 


*w) fix 1 , X 2 , X„) =x 2 + X2 +-“ + x% 

Real-valued function of n real variables 

Function from Rf to 

R 


In this section we shall begin the study of functions of the form w = F(x), 
where the independent variable x is a vector in R n and the dependent variable 
w is a vector in R m . We shall concentrate on a special class of such functions 
called “linear transformations. ” Linear transformations are fundamental in the 
study of linear algebra and have many important applications in physics, 
engineering, social sciences, and various branches of mathematics. 


Functions from rh to r m 

If the domain of a function/is R n and the codomain is R m (m and n possibly the same), then/is called a map or transformation 
from Rf to R™, and we say that the function fmaps R n into R™. We denote this by writing / :R n —* R m . The functions in 
Table 1 are transformations for which m = \. In the case where m = n, the transformation / :R n —* R n is called an operator on 
R n . The first entry in Table 1 is an operator on R. 

To illustrate one important way in which transformations can arise, suppose that / j, f 2 , are real-valued functions of n 

real variables, say 

wi = /lOl. *2 >—>*b) 

W2 = /2(*l.X2 >—>*b) 

Wm = /m(*l.X2 >—>*b) 

These m equations assign a unique point ( Wlj W2i ____ Wjjj ) in R m to each point (x j, xj, x H ) i n R n an d thus define a 










transformation from Rf to R' n . If we denote this transformation by T, then T.R n —* R' n and 

T{x i, * 2 , J n) = (wi, W 2 > 


A Transformation from to ft 3 


The equations 

^ 1=^1 I- *2 

w 2 = 3*1*2 

2 2 
w 3 — ~ x 2 

define a transformation rr 2 _* R?- With this transformation, the image of the point (x\, * 2 ) i § 

T(*i,*2) = (*1 T *2> 3x^X2, xj-xj) 

Thus, for example, R(f - 2) = ( - 1, - 6, - 3) 


♦ 


Linear Transformations from R n to R m 


In the special case where the equations in 1 are linear, the transformation T.R 71 —> R m defined by those equations is called a 
linear transformation (or a linear operator if m = n ). Thus a linear transformation T\R n —* R m is defined by equations of the 
form 


or, in matrix notation, 


or more briefly by 


W1 = <311*1 + <312*2 H-1 <*1h*M 

m>2 = <*21*1 T fl 22*2 + -T 32 h*h 

= <3 m l*l 4- <3 m2 *2 4 - 4 <3 mH *H 


1 

—< CA 
% ? " 


'<311 312 - 

321 322 - 

■“ a l H 

■ ■ 


i 

—< CA 

K * 

i_ 



a m\ a m2 - 



*H 


w = 


( 2 ) 


( 3 ) 


( 4 ) 


The matrix A = is called the standard matrix for the linear transformation T, and T is called multiplication by A. 


A Linear Transformation from R 4 to R 3 


The linear transformation r-r 4 _> r} defined by the equations 

wi = 2*i — 3*2 + *3 — 5*4 

w 2 = 4*i + *2-2*3 +*4 (5) 

W 3 = 5*i— *2 I 4*3 


can be expressed in matrix form as 

















so the standard matrix for T is 


( 6 ) 



*1 

*2 

x 3 

x 4 


The image of a point (x^, xj, x^, X 4 ) can be computed directly from the defining equations 5 or from 6 by matrix multiplication. 
For example, if X 2 , xj, X 4 ) = (1, — 3, 0, 2)> then substituting in 5 yields 

wi = 1, v ?2 = W3 = 8 


(verify) or alternatively from 6, 


~W\~ 


^2 

A = 




1 

-3 
0 
2 

♦ 


1 -5 

-2 1 

4 0 


1 

3 

8 


Some Notational Matters 

If T.R 71 —> R m is multiplication by A, and if it is important to emphasize that A is the standard matrix for T, we shall denote the 
linear transformation T:R n —» R m by Tj±.R n —> R m - Thus 


T a (x)=Ax 


( 7 ) 


It is understood in this equation that the vector x in R n is expressed as a column matrix. 

Sometimes it is awkward to introduce a new letter to denote the standard matrix for a linear transformation T.R H —> R m - In 
such cases we will denote the standard matrix for 7 by the symbol [ 7*1 .With this notation, equation 7 would take the form 

T(x)=\T]x ( 

Occasionally, the two notations for a standard matrix will be mixed, in which case we have the relationship 

[T A ]=A 


Remark Amidst all of this notation, it is important to keep in mind that we have established a correspondence between mxn 
matrices and linear transformations from R n to R m : To each matrix A there corresponds a linear transformation t a 
(multiplication by A), and to each linear transformation T\R n —» R m , there corresponds an mxn matrix [T] (the standard 
matrix for T). 

Geometry of Linear Transformations 

Depending on whether //-tuples are regarded as points or vectors, the geometric effect of an operator T R n _* R n is to 

transform each point (or vector) in R™ into some new point (or vector) (Figure 4.2.1). 

























(y) T maps points Lo points 



(b) T maps vectors to vectors 

Figure 4.2.1 


Zero Transformation from R n to R m 


If 0 is the m x n zero matrix and 0 is the zero vector in R n , then for every vector x in R n , 

7 n U) = 0x = Q 

so multiplication by zero maps every vector in R n into the zero vector in R m . We call R 0 the zero transformation from R n to R m 
. Sometimes the zero transformation is denoted by 0. Although this is the same notation used for the zero matrix, the appropriate 
interpretation will usually be clear from the context. 

♦ 


Identity Operator on R n 


If I is the n x n identity matrix, then for every vector x in R n , 

T](x) =lx = x 

so multiplication by I maps every vector in R n into itself. We call the identity operator on R n . Sometimes the identity 
operator is denoted by I. Although this is the same notation used for the identity matrix, the appropriate interpretation will 
usually be clear from the context. 


Among the most important linear operators on R 2 and RR are those that produce reflections, projections, and rotations. We shall 
now discuss such operators. 


Reflection Operators 










Consider the operator T.R 1 _> B? that maps each vector into its symmetric image about the y-axis (Figure 4.2.2). 



If we let w = T(x), then the equations relating the components of x and w are 

wi = — x = — x -I- Oy 

^2= y= 0x + y 

or, in matrix form, 


~W\- 


' - 1 

o' 

"x' 

^2 


0 

1 _ 

7 


( 10 ) 


(ID 


Since the equations in 10 are linear, T is a linear operator, and from 11 the standard matrix for T is 


In general, operators on g- and that map each vector into its symmetric image about some line or plane are called reflection 
operators. Such operators are linear. Tables 2 and 3 list some of the common reflection operators. 



Table 3 
































Operator Illustration Equations Standard Matrix 


Reflection about the xjy-plane , 

✓ 

L ; wi= x 

W2= y 

Jt :J 3 \ W3 = — z 

/ I _y 

\ —♦-*- 

\i 

M f 

* ft, v, -i) 

'1 0 o' 
0 1 0 

0 0-1 


Reflection about the v—plane ,,, .. , j 

- *- 

w X 

Xg 

rv.v.D w l= 1 

w 2 = —y 

/' W 3 = 2 

jT X Y 

£ - g 

'1 0 o' 

0-10 

0 0 1 


Reflection about the yz-plane , 

X g 

1 “ W1 = — X 

1 {-s.v.zi 

>^2= y 

S* w 3 = z 

> / 

^ y 

'-1 0 o' 
0 1 0 

0 0 1 



Projection Operators 


Consider the operator p-p 2 _> R 2 that maps each vector into its orthogonal projection on the x-axis (Figure 4.2.3). The 

equations relating the components of x and w = T(x') are 

W 1 = X = x -I- 0 .y 
W 2 = 0 = Ox -I- 0.y 


( 12 ) 


or, in matrix form, 


~W\~ 


'1 

o' 

“x “ 

W 2 


_0 

0_ 

y 


(13) 



The equations in 12 are linear, so T is a linear operator, and from 13 the standard matrix for T is 


m 


1 0 
0 0 


In general, a projection operator (more precisely, an orthogonal projection operator ) on p 2 or p} is any operator that maps 
each vector into its orthogonal projection on a line or plane through the origin. It can be shown that such operators are linear. 
Some of the basic projection operators on p} and p} are listed in Tables 4 and 5. 

































Table 4 


Operator Illustration Equations Standard Matrix 

Orthogonal projection on the x-axis , 

l v w i = x 

W2 = 0 

y^\ 

S' tuAh * 

w ^ 

1 1 

o o 

1 — o 

1_1 


Orthogonal projection on the y-axis > 

(0, v) , 

« 

i - v w i = 0 

U- ^2=7 

Jt 

--►" 

1 1 

O T— 

o o 

1_1 



Table 5 


Operator Illustration Equations Standard Matrix 

Orthogonal projection on the xy -plane , 

L - wi=x 

W2 =y 

K / 

y = 0 

■ » (.V, _v. <» 

'1 0 o' 
0 1 0 

0 0 0 


Orthogonal projection on the ^-plane , 

—- 

VI " . 

I y 

z I , W\=X 

W2 = 0 

A v w 3=z 

'1 0 o' 
0 0 0 

0 0 1 


Orthogonal projection on the yz-plane i 

J 

W l=° 

^2 =y 

yr**? W3=z 

^ _ J. 

'0 0 o' 
0 1 0 

0 0 1 



Rotation Operators 

An operator that rotates each vector in S through a fixed angle Q is called a rotation operator on p}. Table 6 gives the formula 
for the rotation operators on R l . To show how this is derived, consider the rotation operator that rotates each vector 
counterclockwise through a fixed positive angle Q. To find equations relating x and w — T’(x), let 6 be the angle from the 
positive x-axis to x, and let r be the common length of x and w (Figure 4.2.4). 































Figure 4.2.4 


Table 6 



Then, from basic trigonometry, 


x = rcos<b, y = rsm6 


and 


uq = r cos(# 4- (b), W 2 = rsm(8 + $) 


(14) 


(15) 


Using trigonometric identities on 15 yields 

uq = r cos 9 cos (b — r sm9 sin fb 
W 2 = r smB cos 6 r cos 0 sin 6 

and substituting 14 yields 

nq = x cos 0 — y s\n8 
Vl>2 = X Sill 0 + 7 cos 9 


(16) 


The equations in 16 are linear, so T is a linear operator; moreover, it follows from these equations that the standard matrix for T 
is 


[T] 


cos 9 — sin Q 

sin 8 cos 9 


EXAMPLE 5 Rotation 


If each vector in r 2 is rotated through an angle of ^ / 6 ( 


30°), then the image w of a vector 
~x " 


X = 


7 


IS 

















cos jt / 6 — sin f 6 

~x " 


fi/2 -1/2 

~x " 

sin?r/ 6 cos W 6 

y 


1/2 j/3/2 

y 


jH 1 

2 X ~2 y 

1 \[[ 
2 XA 2 y 


For example, the image of the vector 



is w = 


g=L 

2 

14 i/3 

2 


A rotation of vectors in p} is usually described in relation to a ray emanating from the origin, called the axis of rotation. As a 
vector revolves around the axis of rotation, it sweeps out some portion of a cone (Figure 4.2.5a). The angle of rotation, which is 
measured in the base of the cone, is described as “clockwise” or “counterclockwise” in relation to a viewpoint that is along the 
axis of rotation looking toward the origin. For example, in Figure 4.2.5a the vector w results from rotating the vector x 
counterclockwise around the axis l through an angle 0. As in p}, angles are positive if they are generated by counterclockwise 
rotations and negative if they are generated by clockwise rotations. 




{/>! Right-hand rule 

Figure 4.2.5 


The most common way of describing a general axis of rotation is to specify a nonzero vector u that runs along the axis of 
rotation and has its initial point at the origin. The counterclockwise direction for a rotation about the axis can then be 
determined by a “right-hand rule” (Figure 4.2.5 b): If the thumb of the right hand points in the direction of u, then the cupped 
fingers point in a counterclockwise direction. 

A rotation operator on p 3 is a linear operator that rotates each vector in p} about some rotation axis through a fixed angle 0. In 
Table 7 we have described the rotation operators on p 3 whose axes of rotation are the positive coordinate axes. For each of 
these rotations one of the components is unchanged by the rotation, and the relationships between the other components can be 
derived by the same procedure used to derive 16. For example, in the rotation about the z-axis, the z-components of x and 
w = T(x) are the same, and the x- and y-components are related as in 16. This yields the rotation equation shown in the last row 
of Table 7. 


Table 7 






















Illustration 


Standard Matrix 


Operator 


Equations 


Counterclockwise rotation about the 
positive x-axis through an angle f) 



= x 

W 2 =y cos# —z sin# 
= y sin #-I-z cos# 


1 0 0 
0 cos 9 —sin 0 
0 sin# cos# 


Counterclockwise rotation about the 
positive y-axis through an angle # 



wi —x cos#-|-zsin# 
™ 2=7 

= — xsin#-| zcos# 


cos # 

0 

sin# 

0 

1 

0 

— sin# 

0 

cos# 


Counterclockwise rotation about the 
positive "-axis through an angle # 



wi —x cos# — y sin# 

W 2 — x sin# 4- y cos # 
wj =z 


cos# —sin# 0 
sin# cos# 0 
0 0 1 


Yaw, Pitch, and Roll 


A I 



Pitch 


In aeronautics and astronautics, the orientation of an aircraft or space shuttle relative to an xyz-coordinate system is often 
described in terms of angles called yaw, pitch, and roll. If, for example, an aircraft is flying along the y-axis and the xy-plane 
defines the horizontal, then the aircraft's angle of rotation about the "-axis is called the yaw, its angle of rotation about the 
x-axis is called the pitch, and its angle of rotation about the y-axis is called the roll. A combination of yaw, pitch, and roll can 
be achieved by a single rotation about some axis through the origin. This is, in fact, how a space shuttle makes attitude 
adjustments—it doesn’t perform each rotation separately; it calculates one axis, and rotates about that axis to get the correct 
orientation. Such rotation maneuvers are used to align an antenna, point the nose toward a celestial object, or position a 
payload bay for docking. 




















For completeness, we note that the standard matrix for a counterclockwise rotation through an angle 9 about an axis in p}, 
which is determined by an arbitrary unit vector u = (a, b, c) that has its initial point at the origin, is 

a^( 1 — cos 9) 4 cos 9 

ab{ 1 — cos 0) + c sin# 

ac{ 1 — cos 9) —bsw.9 

The derivation can be found in the book Principles of Interactive Computer Graphics, by W. M. Newman and R. F. Sproull 
(New York: McGraw-Hill, 1979). The reader may find it instructive to derive the results in Table 7 as special cases of this more 
general result. 

Dilation and Contraction Operators 

If & is a nonnegative scalar, then the operator T(x) = £x on pf or pf is called a contraction with factor k if 0 < k < 1 and a 
dilation with factor k\fk> 1. The geometric effect of a contraction is to compress each vector by a factor of k (Figure 4.2.6a), 
and the effect of a dilation is to stretch each vector by a factor of k (Figure 4.2.6/?). A contraction compresses pf or p} uniformly 
toward the origin from all directions, and a dilation stretches pf or uniformly away from the origin in all directions. 

X 


(a) 0 < k < l 



ab{\ — cos 9) — c sin# ac{\ — cos 9) 4 b sin/? 

b^{\ — cos 0) 4 cos 9 £c(l — cos 9) — a sin# 

2 

Zjc( 1 — cos 9) 4 a sin# c (1 — cos#) 4 cos# 


(17) 



I'iX) — kX 


The most extreme contraction occurs when fc = Q, in which case T(x) = kx reduces to the zero operator T’(x) = 0, which 
compresses every vector into a single point (the origin). If k = 1, then T(x) = kx reduces to the identity operator T(x) = x, 
which leaves each vector unchanged; this can be regarded as either a contraction or a dilation. Tables 8 and 9 list the dilation 
and contraction operators on p 2 and p}. 

Table 8 


Operator 


Illustration 


Equations Standard Matrix 









Operator Illustration Equations Standard Matrix 


Contraction with factor k on (0 < £ < 1) i 

i y W \ = kx 

*jrf >’) 

W2 = ky 

rky} 

X 

~k O' 

Dilation with factor k on p 2 (£ > 1) , 

1 0 k 

i. 1 ’ h di.x. kyi wi=A:x 

pfis.y) w 2 = ky 

J 


Table 9 


Operator 


Illustration Equations Standard Matrix 


Contraction with factor k on p} (0 < k < 1) 


Dilation with factor k on j? 3 (fc > ]) 




C' 


vt?i = kx 
w 2 = ky 
w 2 = kz 


vt?i = kx 
w >7 = ky 
W 3 = kz 


k 

0 

0 


0 0 
k 0 
0 k 


Rotations in ft 3 


Rotation looks 
counterclockwise 



A familiar example of a rotation in p} is the rotation of the Earth about its axis through the North and South Poles. For 
simplicity, we will assume that the Earth is a sphere. Since the Sun rises in the east and sets in the west, we know that the 
Earth rotates from west to east. However, to an observer above the North Pole the rotation will appear counterclockwise, and 
























to an observer below the South Pole it will appear clockwise. Thus, when a rotation in pf> is described as clockwise or 
counterclockwise, a direction of view along the axis of rotation must also be stated. 


There are some other facts about the Earth's rotation that are useful for understanding general rotations in p 2 . For example, as 
the Earth rotates about its axis, the North and South Poles remain fixed, as do all other points that lie on the axis of rotation. 
Thus, the axis of rotation can be thought of as the line of fixed points in the Earth's rotation. Moreover, all points on the 
Earth that are not on the axis of rotation move in circular paths that are centered on the axis and lie in planes that are 
perpendicular to the axis. For example, the points in the Equatorial Plane move within the Equatorial Plane in circles about 
the Earth's center. 


Compositions of Linear Transformations 

If _► R k and p B -p k _> R m are linear transformations, then for each x in R n one can first compute T A {x), which is a 

vector in pf, and then one can compute T s (T A (x)), which is a vector in R m . Thus, the application of p A followed by p B 
produces a transformation from pf to p m . This transformation is called the composition of p B with Ta and is denoted by 
p B o T a ( rea d “Tg circle p A ). Thus 

(T B oT A )(x) = T B (T A (x)) n o 


The composition p B 0 p A is linear since 

(Tp, o T A )(x) = Tp(Ta(x)) = B(Ax) = (BA)x 

s ° T% a T A ' s multiplication by BA, which is a linear transformation. Formula 19 also tells us that the standard matrix for 
p B 0 p A is BA- This is expressed by the formula 

T# o Ta— Tba 


Remat Formula 20 captures an important idea: Multiplying matrices is equivalent to composing the corresponding linear 
transformations in the right-to-left order of the factors. 


There is an alternative form of Formula 20: If Py Pf _, pf and p 2 - p k _> p m are linear transformations, then because the 

standard matrix for the composition p 2 0 T\ is the product of the standard matrices of p 2 and T, we have 

[T 2 oT l ] = [T 2 ][T l ] (? 


Composition of Two Rotations 


Let T’j p 2 _, p 2 and P 2 :R? _* R? be the linear operators that rotate vectors through the angles 0 ^ and () 2 , respectively. Thus 

the operation 

(T 2 o Ti)(x) = T 2 (T 1 (x)) 

first rotates x through the angle 9^, then rotates (x) through the angle 0 2 . It follows that the net effect of p 2 Cl p { is to rotate 
each vector in p 2 through the angle 9 ^ - 0 2 (Figure 4.2.7). 










Figure 4.2.7 


Thus the standard matrices for these linear operators are 

cgs$i — sin$i 


™ = 


sin$i cos0i 

[T 2 oTi] = 


[T 2 ] = 


cos O 2 
sin O 2 


— sin O 2 
cos O 2 


cos(0i 4- & 2 ) ~ sin(0i 4 & 2 ) 

sin(0i 4 02) cos(0i 4 02) 


These matrices should satisfy 21. With the help of some basic trigonometric identities, we can show that this is so as follows: 


[T 2 ][Ti] = 


cos 02 —sin 02 
sin02 cos 02 


cos0i — sin0i 
sin0i cos0i 

cos 02 cos 0i — sin02 sin0i — (cos 02 sin0i 4 sin02 cos 0i 
sin 02 cos 0i 4 cos 02 sin 0i — sin 02 sin 0i 4 cos 02 cos 0i 

cos(0i 4 02) — sin(0i 4 02) 

sin(0i I 02) cos (0i 4 02) 

= [72ori] 


Remark In general, the order in which linear transformations are composed matters. This is to be expected, since the 
composition of two linear transformations corresponds to the multiplication of their standard matrices, and we know that the 
order in which matrices are multiplied makes a difference. 


Composition Is Not Commutative 


Let T\\R 2 _► P? be the reflection operator about the line y=x, and let T^.P 2 _* R 2 be the orthogonal projection on the 

y-axis. Figure 4.2.8 illustrates graphically that T l c , Tj an d T? o T\ have different effects on a vector x. This same conclusion 


can be reached by showing that the standard matrices for 7’ 1 and Tj h° not commute: 

[Ti cT 2 ] = [T 1 ][T 2 ] = 

'o r 
.i °. 

1 1— ' 

O O 

1 _1 

= 

'o r 
_° °. 

[T 2 oTi] = [T 2 ][Ti] = 

'0 o' 
_0 1_ 

'o r 
.i °. 

= 

i i 

o o 

O -1—« 

1 _1 


so [72 o Ti] * [Ti o T 2 ]- 


































ibj T | «r> 


Figure 4.2.8 


Composition of Two Reflections 


Let T\ :R 2 _> P? be the reflection about the y-axis, and let TyR 2 _* P? be the reflection about the x-axis. In this case 

T\ a T 2 an( i T 2 o T\ are the same; both map each vector x = (?:, y) into its negative — x = ( — x, —7) (Figure 4.2.9): 

(Tl oT 2 )(x,y) = 7’i(x, -y) = (-x, -y) 

(T 2 oTi)(x,y)=T 2 (-x,y) = (-x, -y) 




The equality of T\ o T 2 an d T? o T\ can also be deduced by showing that the standard matrices for Tj and r 2 commute: 




















'-1 o' 

'1 

o' 


'-1 

o' 

0 1_ 

_o 

-1_ 


0 

-1_ 

'1 o' 

— 

1 o' 


'-1 

o' 

.0 - i. 


0 1_ 


0 

-1_ 


[TyoT 2 ] = [T 1 ][T 2 ] = 

[T 2 oT { ] = [T 2 \[T { ] = 

The operator T(x) = — x on R 2 or Ry is called the reflection about the origin. As the computations above show, the standard 
matrix for this operator on R 2 is 

" - 1 O' 


[T] = 


0 - 1 


Compositions of Three or More Linear Transformations 

Compositions can be defined for three or more linear transformations. For example, consider the linear transformations 

Tr.R™ >R k , T 2 :R k -► R l , Ty.R 1 * R m 

We define the composition (T 3 o T 2 o T\) :R U —* R m by 

(T 3 oT 2 oT 1 )(x) = T 3 (T 2 (T 1 (x))) 

It can be shown that this composition is a linear transformation and that the standard matrix for R 3 c , J 2 o T\ is related to the 
standard matrices for T\, T 2 , and T 2 by 


[T 3 oT 2 oT i ] = [T 3 ][T 2 ][T l ] 


( 22 ) 


which is a generalization of 21. If the standard matrices for t v 7 ^, and T 2 are denoted by A, B, and C, respectively, then we also 
have the following generalization of 20 : 


Tco Tgo Ta= Tcba 


(23) 


Composition of Three Transformations 


Find the standard matrix for the linear operator T\R? _> R? that first rotates a vector counterclockwise about the z-axis 

through an angle 0, then reflects the resulting vector about the yz-plane, and then projects that vector orthogonally onto the xy 
-plane. 


Solution 

The linear transformation T can be expressed as the composition 

T =T 2 oT 2 oT\ 

where Tj is the rotation about the z-axis, t 2 is the reflection about the yz-plane, and T 2 is the orthogonal projection on the xy 
-plane. From Tables 3, 5, and 7, the standard matrices for these linear transformations are 


cosd — sin^ 0 


1 

1 

O 

O 


1 

0 

0 

sin<? cos^ 0 

. [T 2 ] = 

0 1 0 

. ra = 

0 1 0 

0 0 1 


0 0 1 


0 0 0 


Thus, from 22 the standard matrix for T is [T] = [T 3 ] [T 2 ] [T\ ]; that is, 






























1 

0 

0 

- 1 

0 

0 

cos 9 

— sin 8 

0 

= 

0 

1 

0 

0 

1 

0 

sin 9 

cos 8 

0 


0 

0 

0 

1} 

0 

1 

0 

0 

1 



- COS 8 

sin# 

O' 





= 


sin/? 


cos 9 

0 







0 


0 

0 






Exercise Set 4.2 



Click here for Just Ask! 


Find the domain and codomain of the transformation defined by the equations, and determine whether the transformation is 
!• linear. 


(a) wi = 3*1 — 2*2 4 4*3 
W2 = 5*1 - 8*2 4- x 3 


(b) 

W\ = 

2x1x2 

*2 


^2 = 

*1 

4 - 3 xiX 2 


W 3 = 

*1 

4 - *2 

(c) 

w\ = 

5 xi • 

- x 2 F x 3 


= 

-xi I X2 1 7 x 3 


= 

2 xi ■ 

- 4 x 2 - *2 

(d) 

>t?l = 

A- 

3 x2 4 -X3 — 


J 

W 2 = 3xj — 4*2 — *3 h X 4 


2. 


Find the standard matrix for the linear transformation defined by the equations. 


(a) w i = 2 x i — 3^2 + *4 
W2 = 3*i + 5x2 -*4 


(b) wi = 7xi 4-2x2 — 8 x 3 
= - x 2 4- 5 x 3 

W 3 = 4xi 4-7x2— X 3 














(c) w\ - -JT1 f X 2 
W2= 3x\ — 2x2 
W2= 5xi-1x2 


(d) wi =x\ 

W2 = x\ I X2 

w 3 = xi + X2 4 X 3 

W4 = x\ I X2 4 X 3 I X 4 


Find the standard matrix for the linear operator T P? _> P? given by 

3 . 

wi = 3xi + 5^2 — x 3 
w 2 = 4xi- X 2 +X 2 
W 3 = 3x i 4- 2^2 — ^3 

and then calculate T( — 1, 2, 4) by directly substituting in the equations and also by matrix multiplication. 

Find the standard matrix for the linear operator T defined by the formula. 

4. 


(a) T{x\,X 2 ) = {2x\-X2,x\-\ X2) 

(b) T(x i,x 2 ) = On, * 2 ) 

(c) T(x\,X 2 ,xj) = (x\ + 2x2 I * 3 , *14 5 >X2,xi) 

(d) T{x\,x 2 ,xi) = {4x\,lx2, - 8 x 3 ) 

Find the standard matrix for the linear transformation T defined by the formula. 

5. 


(a) T(x i,x 2 ) = (x 2 , “Xi, xi -I- 3 x 2 , xi -x 2 ) 

(b) T(x\, X2, X3, X4) = ( 7 x 1 I 2 x 2 — X3 + X4, X2 -\ X3, -xi) 

(c) T(x\, x 2 , x 3 ) = (0, 0, 0, 0, 0) 

(d) T{x i,X2, X3, X4) = (X4, xi,X3, X2, xi -X3) 


6. 


In each part, the standard matrix [T] of a linear transformation T is given. Use it to find T(x)- [Express the answers in 
matrix form.] 









In each part, use the standard matrix for T to find T(x ); then check the result by calculating T(x) directly. 

7. 

(a) T{x\,X 2 ) = (-*1 +X 2 ,X 2 );x= (-1,4) 

(b) T(x i, X2, X 3 ) = (2xi — *2 + x 3’ x 2 + x 3’ 0); x = (2, 1, — 3) 

Use matrix multiplication to find the reflection of (-1, 2) about 

8 . 

(a) the x-axis 

(b) they-axis 


(c) the line y = x 


Use matrix multiplication to find the reflection of (2, -5, 3) about 

9. 

(a) the xy-plane 


(b) the v'T-plane 


(c) the yz-plane 



Use matrix multiplication to find the orthogonal projection of (2, -5) on 

10. 

(a) the .x-axis 

(b) the v-axis 

Use matrix multiplication to find the orthogonal projection of (-2, 1, 3) on 

11 . 

(a) the xy-plane 

(b) the x^-plane 

(c) the yz-plane 

Use matrix multiplication to find the image of the vector (3, -4) when it is rotated through an angle of 

12 . 

(a) 0=30° 

(b) 9= -60° 

(c) 9 — 45° 

(d) 0=90° 

Use matrix multiplication to find the image of the vector (-2, 1, 2) if it is rotated 

13. 

(a) 30° about the x-axis 

(b) 45° about the _y-axis 

(c) 90° about the z-axis 

Find the standard matrix for the linear operator that rotates a vector in p} through an angle of _ 60° about 

14. 

(a) the x-axis 


(b) the_y-axis 



(c) the z-axis 


15. 


16. 


Use matrix multiplication to find the image of the vector (-2, 1, 2) if it is rotated 

(a) _ 30° about the x-axis 

(b) _ 45° about the y-axis 

(c) _ 90° about the z-axis 


Find the standard matrix for the stated composition of linear operators on j? 2 . 


(a) A rotation of 90°, followed by a reflection about the line y = x. 


(b) An orthogonal projection on the y-axis, followed by a contraction with factor k = ^-. 


(c) A reflection about the x-axis, followed by a dilation with factor k = 3- 


17. 


Find the standard matrix for the stated composition of linear operators on j? 2 . 


(a) A rotation of 60°, followed by an orthogonal projection on the x-axis, followed by a reflection about the line y 

(b) A dilation with factor fc = 2, followed by a rotation of 45°, followed by a reflection about the y-axis. 

(c) A rotation of 15°, followed by a rotation of 105°, followed by a rotation of 60°. 


Find the standard matrix for the stated composition of linear operators on j? 3 . 

18. 


(a) A reflection about the yz-plane, followed by an orthogonal projection on the ^ z -plane. 

(b) A rotation of 45° about the y-axis, followed by a dilation with factor k = ^2- 

(c) An orthogonal projection on the x y-plane, followed by a reflection about the yz-plane. 



Find the standard matrix for the stated composition of linear operators on j? 3 . 


19. 


(a) A rotation of 30° about the x-axis, followed by a rotation of 30° about the z-axis, followed by a contraction with 

factor k = 4-. 

4 

(b) A reflection about the xy-plane, followed by a reflection about the * z -plane, followed by an orthogonal projection 
on the yz-plane. 

(c) A rotation of 270° about the x-axis, followed by a rotation of 90° about the y-axis, followed by a rotation of 180° 
about the z-axis. 


Determine whether T l 0 T 2 = T 2 o T\- 


(a) T\ .B? _. R l is the orthogonal projection on the x-axis, and T 2 .B? _ * 3? 3 i s the orthogonal projection on the 

y-axis. 

(b) T\ .P? _> R 3 is the rotation through an angle and T 2 :R 2 _* R 2 is the rotation through an angle Q 2 . 

(c) T\ .B? _. r} is the orthogonal projection on the x-axis, and T 2 \B? _> R l is the rotation through an angle Q. 


21 . 


Determine whether 7^ 0 T 2 — T 2 o T\- 


(a) T\ .B? _> B? is a dilation by a factor k, and T 2 .B? _ * B? i s th e rotation about the z-axis 

through an angle 9- 

(b) 7’ 1: B? _> R 3 is the rotation about the x-axis through an angle , and 7 ^ : B? _» B? ' s the rotation about the z-axis 

through an angle Q 2 - 


22. 


In the orthogonal projections on the x-axis, y-axis, and z-axis are defined by 

T\{x,y,z) = (x, 0, 0), T 2 {x,y,z) = {§,y,0), T 2 (x,y,z) = (0, 0,z) 


respectively. 


(a) Show that the orthogonal projections on the coordinate axes are linear operators, and find their standard matrices. 

(b) Show that if T\B? _> B? is an orthogonal projection on one of the coordinate axes, then for every vector x in r} 

the vectors T’(x) and x — T’(x) are orthogonal vectors. 


(c) Make a sketch showing x and x — T(x) in the case where T is the orthogonal projection on the x-axis. 


Derive the standard matrices for the rotations about the x-axis, y-axis, and z-axis in g} from Formula 17. 

23. 

Use Formula 17 to find the standard matrix for a rotation of ^ / 2 radians about the axis determined by the vector 

24. v= (i ; i, i). 

Note Formula 17 requires that the vector defining the axis of rotation have length 1. 

Verify Formula 21 for the given linear transformations. 

25. 

(a) T\{x\, x 2 ) = Ol 4 X 2 , *1 -xj) and 720 i, x 2 ) = (3xi, 2xi H 4x2) 

(b) Ti(xi,x 2 ) = (4xi, -2xi+x 2 , “*l “ 3 x 2 ) and T 2 (xi, x 2 , x 3 ) = (xj, -I- 2x 2 - x 3 , 4xi — x 3 ) 

(c) T’i(xi,x 2 ,X3) = (-X 1 , +x 2 , -X 2 + X3, -X3 4- xj) and T^xi, x 2 , X3) = ( - 2xi, 3 x 3 , “4x 2 ) 


It can be proved that if A is a 2 x 2 matrix with det(j4) = 1 and such that the column vectors of A are orthogonal and have 
26* length 1, then multiplication by A is a rotation through some angle Q . Verify that 


- 1//2 - 1//2 
1//2 - 1//2 


satisfies the stated conditions and find the angle of rotation. 


The result stated in Exercise 26 is also true in g} : It can be proved that if A is a 3 x 3 matrix with det(A) = 1 and such that 
27- the column vectors of A are pairwise orthogonal and have length 1, then multiplication by A is a rotation about some axis of 
rotation through some angle Q. Use Formula 17 to show that if A satisfies the stated conditions, then the angle of rotation 
satisfies the equation 


cos 9 = 


tr(A) - 1 
2 


Let A be a 3 x 3 matrix (other than the identity matrix) satisfying the conditions stated in Exercise 27. It can be shown the 
if x is any nonzero vector in g\ then the vector u = Ax A A T x + [ 1 — tr (A) ] x determines an axis of rotation when u is 

positioned with its initial point at the origin. [See “The Axis of Rotation: Analysis, Algebra, Geometry,” by Dan Kalman, 
Mathematics Magazine , Vol. 62, No. 4, October 1989.] 


I _4 3 

9 9 9 

8 4 1 

9 9 9 

i 7 4 

9 9 9 


(a) Show that multiplication by 



is a rotation. 


(b) Find a vector of length 1 that defines an axis for the rotation. 

(c) Use the result in Exercise 27 to find the angle of rotation about the axis obtained in part (b). 


Discussion 

In words, describe the geometric effect of multiplying a vector x by the matrix A. 


(a) 



0 

0 


(b) 




In words, describe the geometric effect of multiplying a vector x by the matrix A. 

30 . 


(a) 



0 

3 


(b) 


A = 


{in -i/2 

1/2 ^ 3/2 


31 . 


In words, describe the geometric effect of multiplying a vector x by the matrix 

cos 2 #— sin 2 # —2 sin#cos# 

2 sin#cos# cos 2 # —sin 2 # 


If multiplication by A rotates a vector x in the xy-plane through an angle #, what is the effect of 
32 . multiplying x by a Explain your reasoning. 

Let xg be a nonzero column vector in p}, and suppose that T\P? _> K? is the transformation 

33* defined by T(x) = xg | Rqx, where R f/ is the standard matrix of the rotation of p} about the origin 
through the angle #. Give a geometric description of this transformation. Is it a linear 
transformation? Explain. 

A function of the form j (*) = mx b is commonly called a “linear function” because the graph 
34 * of y = mx A h >s a line. Is /a linear transformation on R7 

Let x = xg I tv be a line in R n , and let T\ R n —* R n be a linear operator on R n . What kind of 
33 - geometric object is the image of this line under the operator 77 Explain your reasoning. 
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4.3 

PROPERTIES OF LINEAR 
TRANSFORMATIONS 
FROM R" TO R m 


In this section we shall investigate the relationship between the invertibility of a 
matrix and properties of the corresponding matrix transformation. We shall also 
obtain a characterization of linear transformations from R n to R m that will form 
the basis for more general linear transformations to be discussed in subsequent 
sections, and we shall discuss some geometric properties of eigenvectors. 


One-to-One Linear Transformations 

Linear transformations that map distinct vectors (or points) into distinct vectors (or points) are of special importance. One example 

of such a transformation is the linear operator x.R 2 _+ R 2 that rotates each vector through an angle Q. It is obvious 

geometrically-that if u and v are distinct vectors in p A , then so are the rotated vectors T(\\) and T(y) (Figure 4.3.1). 


X 



Figure 4.3.1 

Distinct vectors u and v are rotated into distinct vectors T(\\) and T(y) ■ 


In contrast, if T\R? _► R? is the orthogonal projection of p}' on the xy-plane, then distinct points on the same vertical line are 

mapped into the same point in the xy-plane (Figure 4.3.2). 



Figure 4.3.2 

The distinct points P and Q are mapped into the same point M. 


DEFINITION 


A linear transformation T- R y} >R m 

(points) in R m . 

is said to be one-to-one if T maps distinct vectors (points) in into distinct vectors 


Remark It follows from this definition that for each vector w in the range of a one-to-one linear transformation T, there is exactly 
one vector x such that T(x) = w- 
















One-to-One Linear Transformations 


In the terminology of the preceding definition, the rotation operator of Figure 4.3.1 is one-to-one, but the orthogonal projection 
operator of Figure 4.3.2 is not. 

♦ 


Let A be an n x « matrix, and let T^.R n —> R n be multiplication by A. We shall now investigate relationships between the 
invertibility of A and properties of r a . 

Recall from Theorem 2.3.6 (with w in place of b) that the following are equivalent: 


A is invertible. 


= w is consistent for every n x 1 matrix w. 


= w has exactly one solution for every BX 1 matrix w. 

However, the last of these statements is actually stronger than necessary. One can show that the following are equivalent (Exercise 
24): 


A is invertible. 


j4x = w is consistent for every n x 1 matrix w. 


= w has exactly one solution when the system is consistent. 


Translating these into the corresponding statements about the linear operator r a , we deduce that the following are equivalent: 

A is invertible. 

For every vector w in R n , there is some vector x in R n such that T^(x) = w- Stated another way, the range of R A is all of R n . 

For every vector w in the range of J A , there is exactly one vector x in R n such that T^(x) = w- Stated another way, J A is 
one-to-one. 

In summary, we have established the following theorem about linear operators on R n . 


THEOREM 4.3.1 


Equivalent Statements 

If A is an n x n matrix and Tj±.R n -> R n is multiplication by A, then the following statements are equivalent. 







(a) A is invertible. 


(b) The range of is R n . 


(c) R^ is one-to-one. 


Applying Theorem 4.3.1 


In Example 1 we observed that the rotation operator T.R? _► R? illustrated in Figure 4.3.1 is one-to-one. It follows from 

Theorem 4.3.1 that the range of T must be all of R 2 and that the standard matrix for T must be invertible. To show that the range of 
T is all of r}, we must show that every vector w in R? is the image of some vector x under T. But this is clearly so, since the vector 
x obtained by rotating w through the angle — # maps into w when rotated through the angle 0. Moreover, from Table 6 of Section 
4.2, the standard matrix for T is 


[T] 


cos# —sin# 
sin# cos# 


which is invertible, since 


det [T] - 


cos # 
sin# 


— sin# 
cos # 


= cos 2 # -I- sin 2 # = 1 3* 0 


♦ 


Applying Theorem 4.3.1 


In Example 1 we observed that the projection operator T.R? _> R? illustrated in Figure 4.3.2 is not one-to-one. It follows from 

Theorem 4.3.1 that the range of T is not all of R? and that the standard matrix for T is not invertible. To show directly that the 
range of T is not all of R 3 , we must find a vector w in R^ that is not the image of any vector x under T. But any vector w outside of 
the xy-plane has this property, since all images under T lie in the xy-plane. Moreover, from Table 5 of Section 4.2, the standard 
matrix for T is 


[T] 


1 0 0 
0 1 0 
0 0 0 


which is not invertible, since det[ T] = 0- 


Inverse of a One-to-One Linear Operator 

If T_^.R n —* R n is a one-to-one linear operator, then from Theorem 4.3.1 the matrix A is invertible. Thus, T^_i :Rf -> R n is 

itself a linear operator; it is called the inverse of r a The linear operators r a and T \ cancel the effect of one another in the sense 
that for all x in Rf, 

T j 4 (T j 4 _i(x))=AA- 1 x = /x = x 
T j 4 _i(T j 1 (x))=A" 1 Ax = /x = x 


or, equivalently, 
















t a-'° Ta = t a-'a = t ‘ 

From a more geometric viewpoint, if w is the image of x under r a , then 7” 1 maps w back into x, since 

T^_i(w) = ^_i(T j4 ( X ))=x 

(Figure 4.3.3). 



Before turning to an example, it will be helpful to touch on a notational matter. When a one-to-one linear operator on R n is written 

as T:R n _+ R n (rather than Tj^.R n —* R n ), then the inverse of the operator T is denoted by 7 -1 (rather than T^-i). Since the 

standard matrix for 7’ -1 is the inverse of the standard matrix for T, we have 

[T~' l ] = [T]-' 1 ( 1 ) 


EXAMPLE 4 Standard Matrix for r _1 


Let J\R? 


r} be the operator that rotates each vector in r} through the angle #, so from Table 6 of Section 4.2, 


[T] 


cos 8 — sin# 

sin# cos# 


( 2 ) 


It is evident geometrically that to undo the effect of T, one must rotate each vector in R 1 through the angle — #. But this is exactly 
what the operator 7’ -1 does, since the standard matrix for 7’ -1 is 


[T~ 1 ] = [T] 


1 _ 

cos# 

sin# 


cos( — 8 ) 

S'" 

1 

Oj 

1 


— sin# 

cos# 


sin( — #) 

cos( — 8 ) 


(verify), which is identical to 2 except that # is replaced by — #. 


EXAMPLE 5 Finding T -1 

Show that the linear operator T\R 2 _> R? defined by the equations 

w 1 = 2x 1 4- X 2 
w 2 = 3xi + 4*2 

is one-to-one, and find 7 -1 ( W1> W2 )- 


Solution 


The matrix form of these equations is 

















so the standard matrix for T is 


"Wl" 


'2 

r 

‘*f 

w 2 


_3 

4_ 

/ 2 _ 


[T] = 


2 1 
3 4 


This matrix is invertible (so T is one-to-one) and the standard matrix for 7 ’ -1 is 


Thus 


from which we conclude that 


[T~ l ] = [T]- l = 


4 

5 
3 
5 


[T~ l ] = 




4 r 


~W\ " 


5 5 


~W\~ 

w 2 


3 2 


w 2 



5 5 



-wi --w 2 


.-1 


4. 

5 

3 2 

-wi -w 2 


(wi. = (jwi - jw 2 , - + |w 2 ) 


Linearity Properties 

In the preceding section we defined a transformation T.R n -* R m to be linear if the equations relating x and w = T(\) are linear 

equations. The following theorem provides an alternative characterization of linearity. This theorem is fundamental and will be the 
basis for extending the concept of a linear transformation to more general settings later in this text. 


THEOREM 4.3.2 

Properties of Linear Transformations 

A transformation T.R n —» R m is linear if and only if the following relationships hold for all vectors u and v in R n and for 
every scalar c. 

(a) T(\\ |- v) = T{ u) + T(y) 

(b) T(cu) = cT(u) 


Proof Assume first that / is a linear transformation, and let A be the standard matrix for T. It follows from the basic arithmetic 
properties of matrices that 


T(u -I- v) = A(u 4- v) = j4u H j4v = T (u) -|- T (v) 


and 


T(cu) =j4(cu) = c (Au) =c7’(u) 


Conversely, assume that properties (a) and (£>) hold for the transformation T. We can prove that T is 
linear by finding a matrix A with the property that 























T(x) = At 


( 3 ) 


for all vectors x in r”. This will show that T is multiplication by A and therefore linear. But before we 
can produce this matrix, we need to observe that property (a) can be extended to three or more terms; 
for example, if u, v, and w are any vectors in R n , then by first grouping v and w and applying property 
(a), we obtain 

T{ u + v + w) = T(n + (v + w)) = T{ u) f T(v + w) = T(u) + T(v) f T(w) 

More generally, for any vectors vi, v 2 , v* in £ H , we have 

T(vi + Y 2 4- - 4- Yjfc) = r(vi) + T(y 2 ) + - 4- T(v k ) 

Now, to find the matrix A, let ei, <? 2 / ■■■, be the vectors 


V 


Y 


Y 

0 


i 


0 

0 

- *2 = 

0 

=■ — 

0 

0 


0 


l 


and let A be the matrix whose successive column vectors are T(e 0, T(<? 2 )/ ■■■/ T(e„); that is, 


^=[T(ei)|T(e 2 )|-|T(e H )] 


(5) 


If 

‘*r 

*2 

x = . 

is any vector in R n , then as discussed in Section 1.3, the product Ax is a linear combination of the 
column vectors of A with coefficients from x, so 

^x = xi7(ei) + x 2 T(e 2 ) + - + 

= rOiei) + T(x 2 <? 2 ) + + Property ( b ) 

= T(;riei + ^2^2 H-1- x yi^yi) Property (b) for n terms 

= T(x) 

which completes the proof. 

■ 

Expression 5 is important in its own right, since it provides an explicit formula for the standard matrix of a linear operator 
T.R M —* R m in terms of the images of the vectors ep e 2 , under T. For reasons that will be discussed later, the vectors ep 

e 2 , .e,, in 4 are called the standard basis vectors for R n . In R 1 and R s these are the vectors of length 1 along the coordinate axes 
(Figure 4.3.4). 
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Figure 4.3.4 


Because of its importance, we shall state 5 as a theorem for future reference. 


THEOREM 4.3.3 


IfT.Rf —» R m is a linear transformation, and ©p *2> •••> are the standard basis vectors for R n , then the standard matrix 
for T is 

[T] = [T(ei)\n*2)\-‘-\TM] (f 


Formula 6 is a powerful tool for finding standard matrices and analyzing the geometric effect of a linear transformation. For 

example, suppose that T.R? _* i? 3 is the orthogonal projection on the xy-plane. Referring to Figure 4.3.4, it is evident 

geometrically that 


so by 6, 



V 


'O' 


'O' 

r(ei) = ei = 

0 

to 

II 

'T 

to 

II 

1 

* 

LO- 

II 

o 

II 

0 


0 


0 


0 


[T] 


1 0 0 
0 1 0 
0 0 0 


which agrees with the result in Table 5 of Section 4.2. 


Using 6 another way, suppose that T&R? 


pf is multiplication by 


A = 


-1 2 1 
3 0 6 


The images of the standard basis vectors can be read directly from the columns of the matrix A: 






















Ta 


i_ n 

0 


1 / 

To' 

■i 


" ";i ~ 


I 0 ’ 


t 

_ 6 _ 

’ ta \ 

l 

1° 


— 

_0_ 

- Ta 

0 

[l 

] = 


Standard Matrix for a Projection Operator 


Let l be the line in the xy-plane that passes through the origin and makes an angle 0 with the positive v-axis, where 0 < 9 < it. As 
illustrated in Figure 4.3.5a, let T.P? _> P? be a linear operator that maps each vector into its orthogonal projection on /. 





(el 

Figure 4.3.5 


(a) Find the standard matrix for T. 


(b) Find the orthogonal projection of the vector x = (1, 5) onto the line through the origin that makes an angle of Q = ^ / 6 with 
the positive v-axis. 


Solution (a) 




























From 6, 


[T] = [T(ei)|T(e 2 )] 

where cj and e 2 are the standard basis vectors for p}. We consider the case where 0 <8 <x / 2; the case where - / 2 < 9< x is 
similar. Referring to Figure 4.3.5b, we have HTfej) || = cos 9, so 

T(e 0 = 


T(9. 2 ) = 


'||7Xei)||cos0' 


cos 2 # 

||7Xei)||sin# 


sin# cos # 

i, so 

" l|7’(e 2 ) || cos #~ 


sin# cos # 

l|7’(e 2 ) ||sin# 


sin 2 # 


Thus the standard matrix for T is 


[T] = 


2 

cos & sin# cos# 
sin 9 cos 9 sin 2 # 


Solution (b) 

Since sin / 6 = 1 / 2 and cos jt / 6 = i/J/ 2> ^ follows from part (a) that the standard matrix for this projection operator is 

3/4 /3/4 


[T} = 


j/3/4 1/4 


Thus 


ff'i 


3/4 

^3/4 

'1 

hi 

r 

^T 

1 / 4 

_5 


3 + 5^3 
4 

l/3 + 5 


or, in point notation, 


7X1,5) = 


/ 3 + 5(/3 J3 + 5 
[4-4 


Geometric Interpretation of Eigenvectors 

Recall from Section 2.3 that if A is an n x n matrix, then \ is called an eigenvalue of A if there is a nonzero vector x such that 

j4x = Ax or, equivalently, (A I — j4)i = 0 

The nonzero vectors x satisfying this equation are called the eigenvectors of A corresponding to A- 

Eigenvalues and eigenvectors can also be defined for linear operators on p n \ the definitions parallel those for matrices. 


DEFINITION 


If T.R n —► R n is a linear operator, then a scalar \ is called an eigenvalue of T if there is a nonzero x in p n such that 

T(x) = Ax 

Those nonzero vectors x that satisfy this equation are called the eigenvectors ofT corresponding to \. 

































Observe that if A is the standard matrix for T, then 7 can be written as 

= Ax 


from which it follows that 


The eigenvalues of T are precisely the eigenvalues of its standard matrix A. 


x is an eigenvector of T corresponding to A if and only if x is an eigenvector of A corresponding to A- 


If A is an eigenvalue of A and x is a corresponding eigenvector, then j4 x = Ax. so multiplication by A maps x into a scalar multiple 
of itself. In p} and this means that multiplication by A maps each eigenvector x into a vector that lies on the same line as x 
(Figure 4.3.6). 



(«} A >6 





<*) A SO 

Figure 4.3.6 


Recall from Section 4.2 that if A > 0, then the linear operator Alx = Ax compresses x by a factor of A if 0 < A < 1 or stretches x by a 
factor of A if A > 1. If A <. 0. then Ax = Ax reverses the direction of x and compresses the reversed vector by a factor of |A| if 
0 < |A| < 1 or stretches the reversed vector by a factor of |A| if |A| > 1 (Figure 4.3.7). 



Ui) OSASl 
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(c) -I < A < 0 


Ul) AS-! 


Figure 4.3.7 


Eigenvalues of a Linear Operator 


Let T- p 2 _> 


p} be the linear operator that rotates each vector through an angle Q. It is evident geometrically that unless 9 is a 




multiple of x, T does not map any nonzero vector x onto the same line as x; consequently, T has no real eigenvalues. But if # is a 
multiple of x, then every nonzero vector x is mapped onto the same line as x, so every nonzero vector is an eigenvector of T. Let us 
verify these geometric observations algebraically. The standard matrix for T is 


A = 


cos 9 
sin 0 


— sin# 
cos 9 


As discussed in Section 2.3, the eigenvalues of this matrix are the solutions of the characteristic equation 


det(A/ — A) 


A — cos# 
— sin# 


sin# 

A — cos# 


= 0 


that is, 


(A — cos #)^ 4- sin^# = 0 


( 8 ) 


But if # is not a multiple of x, then sin 2 y > o, so this equation has no real solution for \, and consequently A has no real 
eigenvalues.* If 9 is a multiple of then sin# = 0 and either C os # = 1 or C os # = — 1, depending on the particular multiple of x- In 

"i 

the case where sin# = 0 and cos # = 1, the characteristic equation 8 becomes (A — 1) = 0, so \ = 1 is the only eigenvalue of A. In 
this case the matrix A is 


A = 


1 0 
0 1 


= / 


Thus, for all x in g}, 

T{x) = Ax = lx = x 

so T maps every vector to itself, and hence to the same line. In the case where sin# = 0 and C os # = — 1, the characteristic equation 
8 becomes (A + l) 2 = 0, so \ = — 1 is the only eigenvalue of A. In this case the matrix A is 


A = 


-1 0 
0 - 1 




Thus, for all x in 

T(x) = 2lx = — lx = —x 

so T maps every vector to its negative, and hence to the same line as x. 


EXAMPLE 8 Eigenvalues of a Linear Operator 


Let T.F? _ * B? be the orthogonal projection on the xy-plane. Vectors in the xy-plane are mapped into themselves under T, so 

each nonzero vector in the xy-plane is an eigenvector corresponding to the eigenvalue A = 1 • Every vector x along the z-axis is 
mapped into 0 under T, which is on the same line as x, so every nonzero vector on the "-axis is an eigenvector corresponding to the 
eigenvalue A = 0- Vectors that are not in the xy-plane or along the "-axis are not mapped into scalar multiples of themselves, so 
there are no other eigenvectors or eigenvalues. 


To verify these geometric observations algebraically, recall from Table 5 of Section 4.2 that the standard matrix for T is 


A = 


1 0 
0 1 
0 0 


0 

0 

0 


The characteristic equation of A is 


det(A/ — A) 


A-1 0 

0 A-1 
0 0 


= 0 or 


(A — 1) 2 A = 0 


which has the solutions A = 0 and A = 1 anticipated above. 


As discussed in Section 2.3, the eigenvectors of the matrix A corresponding to an eigenvalue A are the nonzero solutions of 
















( 9 ) 


'A - 1 

0 

O' 

■*r 


'O' 

0 

A- 1 

0 

*2 

= 

0 

0 

0 

A 

x 3 


0 


If \ = 0 , this system is 


'-1 

0 

o' 

■*f 


'o' 

0 

- 1 

0 

x 2 

= 

0 

0 

0 

0 

x 3 


0 


which has the solutions x j = 0 , = 0 > 13 = t (verify), or, in matrix form, 


■*f 


'o' 

*2 

= 

0 

*3 


t 


As anticipated, these are the vectors along the z-axis. If \ = 1 , then system 9 is 


'0 0 o' 

■*l' 


'o' 

0 0 0 

*2 

= 

0 

1- 

o 

o 

1_ 

x 3 


0 


which has the solutions x\=s, X 2 =t’ x 3 = 0 (verify), or, in matrix form, 


"*r 


~s " 

*2 

= 

£ 

x 3 


0 


As anticipated, these are the vectors in the xy-plane. 


♦ 


Summary 

In Theorem 2.3.6 we listed six results that are equivalent to the invertibility of a matrix A. We conclude this section by merging 
Theorem 4.3.1 with that list to produce the following theorem that relates all of the major topics we have studied thus far. 


THEOREM 4.3.4 


Equivalent Statements 

If A is an n matrix, and if Ta R n —> Rf is multiplication by A, then the following are equivalent. 

(a) A is invertible. 


(b) j4x =0 has only the trivial solution. 


(c) The reduced row-echelon form of A is / . 


(d) A is expressible as a product of elementary matrices. 


(e) = h is consistent for every n x 1 matrix b. 


(f) Ax = ti h as exactly one solution for every n x 1 matrix b. 






























(g) det(j4) 0- 

(h) The range of Tj\ is R n . 

(i) Tj\ is one-to-one. 

Exercise Set 4.3 



By inspection, determine whether the linear operator is one-to-one. 



(a) the orthogonal projection on the x-axis in p} 


(b) the reflection about the v-axis in pf 


(c) the reflection about the line y = x in pf 

(d) a contraction with factor k > 0 in R 2 


(e) a rotation about the "-axis in pf 


(f) a reflection about the xy-plane in pf 


(g) a dilation with factor fc > Q in p 3 


Find the standard matrix for the linear operator defined by the equations, and use Theorem 4.3.4 to determine whether the 



operator is one-to-one. 


(a) = 8xi 4- 4*2 

W2 = 2xi4- X2 


(b) u?i = 2xi — 3x2 
W2 = 5xi F X2 





(c) wi = — *1 f 3x2 -I 2*3 
W 2 = 2x\ H 4*3 

W 3 = f3* 2 H 6*3 


(d) wi = *i f 2*2 + 3*3 
vi?2 = 2*i f 5*2 1-3*3 
W 3 = *i -I- S *3 


Show that the range of the linear operator defined by the equations 

=4*i - 2*2 
w 2 = 2 *i- *2 

is not all of and find a vector that is not in the range. 

Show that the range of the linear operator defined by the equations 
wi = *i — 2*2 + *3 
W 2 = 5*i -*2 + 3*3 
W3 = 4*1 + *2 4 2*3 

is not all of j? 3 , and find a vector that is not in the range. 


Determine whether the linear operator 'YP? _> p} defined by the equations is one-to-one; if so, find the standard matrix for 

the inverse operator, and find T -1 (v v\,W 2 )- 


(a) wi = *i 1 - 2*2 

W 2 = — *i + *2 

(b) wi = 4*i — 6*2 

W2 = - 2 *i + 3*2 

(c) wi = -* 2 
w 2 = -*i 

(d) wj = 3*i 

W2 = - 5*i 


Deter mi ne whether the linear operator T.P? 


the inverse operator, and find 7 ,- 1 (wi,W 2 ,W 3 ) 


p} defined by the equations is one-to-one; if so, find the standard matrix for 


(a) wi = *i — 2*2 + 2*3 

w 2 = 2*i+ *2 + *3 
W2= xi f *2 




(b) wi = xi —3x2-1 4*3 
w 2 = -x\ x 2 + *3 

W 3 = — 2x2 4 5x 3 


(c) wi = xi f 4x2 — *3 

m >2 = 2xi + 7*2 + *3 
W3= xi f 3x2 


(d) wi = xi 4- 2x2 + *3 
w>2 = — 2xi + *2 + 4x3 
vi >3 = 7xi + 4x2 — 5x3 


By inspection, determine the inverse of the given one-to-one linear operator. 

7- 


(a) the reflection about the x-axis in p} 


(b) the rotation through an angle of jr / 4 in p 2 


(c) the dilation by a factor of 3 in p 2 


(d) the reflection about the yz-plane in p- 


( e ) the contraction by a factor of ^ in p} 


In Exercises 8 and 9 use Theorem 4.3.2 to determine whether t.F? 


p 2 is a linear operator. 


(a) T{x,y) = {2x,y) 

(b) T{x,y) = {x 2 ,y) 


(c) T{x,y) = {-y,x) 

(d) T(x,7) = (*,0) 


9 . 


(a) T(x,y) = (2x I y,x —y) 


(b) T{x,y) = {x f 1,^) 


(c) T{x,y) = {y,y) 

(d) T(x,y) = ^,Yy) 

In Exercises 10 and 11 use Theorem 4.3.2 to determine whether T R-' _. g^ is a linear transformation. 

10 . 

(a) T{x,y,z) = (x,x + y I z) 

(b) T(x,y,z) = ( 1, 1) 

11 . 

(a) T(x,y,z) = ( 0,0) 

(b) T{x,y,z) = {2x-Ay,2x-5z) 

In each part, use Theorem 4.3.3 to find the standard matrix for the linear operator from the images of the standard basis 

12. vectors. 

(a) the reflection operators on g} in Table 2 of Section 4.2 

(b) the reflection operators on g} in Table 3 of Section 4.2 

(c) the projection operators on g} in Table 4 of Section 4.2 

(d) the projection operators on g 3 in Table 5 of Section 4.2 

(e) the rotation operators on g 2 in Table 6 of Section 4.2 

(f) the dilation and contraction operators on g} in Table 9 of Section 4.2 

Use Theorem 4.3.3 to find the standard matrix for T P? _> P? from the images of the standard basis vectors. 

13. 

(a) T.P? _. P? projects a vector orthogonally onto the x-axis and then reflects that vector about the v-axis. 



(b) T.R 1 —* P? reflects a vector about the line y = x and then reflects that vector about the v-axis. 


(c) T p} _j p} dilates a vector by a factor of 3, then reflects that vector about the line y = x, and then projects that 

vector orthogonally onto the y-axis. 


14. 


Use Theorem 4.3.3 to find the standard matrix for pp 3 _> p} from the images of the standard basis vectors. 


(a) pp} _. p^ reflects a vector about the ^-plane and then contracts that vector by a factor of -j. 

(b) T.F? _ * B? projects a vector orthogonally onto the * z -plane and then projects that vector orthogonally onto the xy 

-plane. 


(c) T\B? _* P? reflects a vector about the xy-plane, then reflects that vector about the x Z -plane, and then reflects that 

vector about the yz-plane. 


15. 


Let p A p- 


pi be multiplication by 


A = 


-13 0 

2 1 2 

4 5-3 


and let ej, ^ ant l be the standard basis vectors for p}'. Find the following vectors by inspection. 


(a) T A {%\), T a (? 2 )’ and 71403) 

(b) T A (e 1 + ?2 + e 3) 

(c) T A <7ej) 


Determine whether multiplication by A is a one-to-one linear transformation. 


(a) 


A = 


- 1 

0 

-4 


M A = 


1 2 

-1 0 


3 

-4 


16 . 



(c) 


'l 2 1 

0 1 1 

1 1 0 

10-1 

Use the result in Example 6 to find the orthogonal projection of x onto the line through the origin that makes an angle 0 with 

17. the positive x-axis. 

(a) x = (- 1, 2);0 = 45° 

(b) x = (1, 0); 0= 30° 

(c) x = (1, 5); 9= 120° 

Use the type of argument given in Example 8 to find the eigenvalues and corresponding eigenvectors of T. Check your 

18. conclusions by calculating the eigenvalues and corresponding eigenvectors from the standard matrix for T. 

(a) T.F? _ * F? is the reflection about the x-axis. 

(b) T.F? _ * F? is the reflection about the line y = x- 


(c) T.F? _ * F? is the orthogonal projection on the x-axis. 

(d) T.F? _* F? is the contraction by a factor of 


Follow the directions of Exercise 18. 

19. 

(a) T.F? _ * i? 3 is the reflection about the yz- plane. 


(b) T.F? _* i? 3 is the orthogonal projection on the xz-plane. 


(c) t. F? _»i? 3 is the dilation by a factor of 2. 


(d) T. F? _* i? 3 is a rotation of tt / 4 about the --axis. 



20 . 

(a) Is a composition of one-to-one linear transformations one-to-one? Justify your conclusion. 

(b) Can the composition of a one-to-one linear transformation and a linear transformation that is not one-to-one be 
one-to-one? Account for both possible orders of composition and justify your conclusion. 

Show that T{x, 7 ) = (0, 0) defines a linear operator on p2 but T{x, j?) = ( 1 , 1 ) does not. 


22 . 

(a) Prove that if T.R n —* R m is a linear transformation, then Tip) = 0—that is, T maps the zero vector in R n into the 
zero vector in R m . 

(b) The converse of this is not true. Find an example of a function that satisfies 7 ( 0 ) = 0 but is not a linear transformation. 

Let / be the line in the % y-plane that passes through the origin and makes an angle 0 with the positive x-axis, where 0 < 9 < x. 
23. Let t p 2 _> p 2 be the linear operator that reflects each vector about / (see the accompanying figure). 


(a) Use the method of Example 6 to find the standard matrix for T. 

(b) Find the reflection of the vector x = (1, 5) about the line / through the origin that makes an angle of 0 = 30 c with the 
positive .r- axis. 



Prove: An n x n matrix A is invertible if and only if the linear system ^x = w has exactly one solution for every vector w in 
24. for which the system is consistent. 


Discussion 

Discovery 


Indicate whether each statement is always true or sometimes false. Justify your answer by giving a 
logical argument or a counterexample. 


(a) If T maps R n into R m , and 7(0) = 0> then T is linear. 








(b) If T.R H —» R m is a one-to-one linear transformation, then there are no distinct vectors u 
and v in R n such that T’fu _ v) = 0- 

(c) If T\ R n —* R n is a linear operator, and if T’(x) = 2x for some vector x, then A = 2 is an 
eigenvalue of T. 

(d) If T maps R n into R m , and if T(ciu -I- c 2 v) =c { T(\\) 4- c 2 7(v) for all scalars cj and c 2 and 
for all vectors u and v in R n , then T is linear. 


Indicate whether each statement is always true, sometimes true, or always false. 

26. 

(a) If T.R H —» R m is a linear transformation and m>n, then T is one-to-one. 

(b) If T.R H —» R m is a linear transformation and m < n , then T is one-to-one. 

(c) If T.R H —» R m is a linear transformation and m = n > then T is one-to-one. 


Let A be an n x n matrix such that det(j4) = Cf and let T R n —> R n be multiplication by A. 

27. 


(a) What can you say about the range of the linear operator 77 Give an example that illustrates 
your conclusion. 

(b) What can you say about the number of vectors that T maps into 0? 


In each part, make a conjecture about the eigenvectors and eigenvalues of the matrix A 
28. corresponding to the given transformation by considering the geometric properties of multiplication 
by A. Confirm each of your conjectures with computations. 


(a) Reflection about the line y = c- 


(b) Contraction by a factor of -i. 
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4.4 

LINEAR 

TRANSFORMATIONS 
AND POLYNOMIALS 


In this section we shall apply our new knowledge of linear transformations to 
polynomials. This is the beginning of a general strategy of using our ideas about 
R n to solve problems that are in different, yet somehow analogous, settings. 


Polynomials and Vectors 

Suppose that we have a polynomial function, say 

2 

p(x) =ax 4- bx -I- c 

where x is a real-valued variable. To form the related function 2p(x) we multiply each of its coefficients by 2: 

2 p(x) = 2ax^ 4 2 bx 4 2c 

That is, if the coefficients of the polynomial p(x) are a, b, c in descending order of the power of x with which they are associated, 
then 2p{x) is also a polynomial, and its coefficients are 2a, 2b, 2c in the same order. 

“i 

Similarly, if q{x) = dx 4 - ex-Y j is another polynomial function, then p(x) y q(x') is also a polynomial, and its coefficients are 
a 4 d, b + e, c 4- /. We add polynomials by adding corresponding coefficients. 

This suggests that associating a polynomial with the vector consisting of its coefficients may be useful. 


Correspondence between Polynomials and Vectors 


Consider the quadratic function p(x) = ax 2 + bx 4- c. Define the vector 


z = 


consisting of the coefficients of this polynomial in descending order of the corresponding power of x. Then multiplication of p(x) 
by a scalar s gives sp(x) = sax 4- sbx 4- sc, and this corresponds exactly to the scalar multiple 


sz = 


sa 

sb 

sc 


of z■ Similarly, p(x) p(x) is 2ax l 4 2 bx -V 2c, and this corresponds exactly to the vector sum z | z : 



~a~ 


"lJ " 

z -+ z = 

b 

+ 

b 


c 


c 


2 a 
2b 
2c 


In general, given a polynomial p ( x ) = a „x n y a H _ix M_1 4- + a\x + a 0 


we associate with it the vector 
















z = 


a n 
a n —1 


*1 

aQ 

in J S M+1 (Figure 4.4.1). It is then possible to view operations like p(x) -> 2p(x) as being equivalent to a linear transformation on 

j? H+1 , namely T(x) = 2z- We can perform the desired operations in p n+1 rather than on the polynomials themselves. 

>3 K"* ■ 


z 



Figure 4.4.1 

The vector z is associated with the polynomial p. 


Addition of Polynomials by Adding Vectors 


Let p(x) = 4x 3 — 2x 4- 1 and q{x) = 3x 3 — 3x 2 4- x. Then to compute r (x) = 4p{x) — 2q(x), we could define 


4' 


3' 

0 

, Y = 

-3 

-2 

1 

1 


0 


and perform the corresponding operation on these vectors: 



4' 


3' 

4u — 2v = 4 

0 

-2 

-2 

-3 

1 


1 


0 


10 

6 

- 10 
4 

Hence r(x) = 10x 3 4- 6x 2 — 10tt 4- 4. 


♦ 


This association between polynomials of degree n and vectors in would be useful for someone writing a computer program 
to perform polynomial computations, as in a computer algebra system. The coefficients of polynomial functions could be stored as 
vectors, and computations could be performed on these vectors. 


For convenience, we define p n to be the set of all polynomials of degree at most n (including the zero polynomial, all the 
coefficients of which are zero). This is also called the space of polynomials of degree at most n. The use of the word space 
indicates that this set has some sort of structure to it. The structure of p n will be explored in Chapter 8. 


Differentiation of Polynomials 




















Calculus Required 


Differentiation takes polynomials of degree n to polynomials of degree n — 1, so the corresponding transformation on vectors must 
take vectors in ^w+1 to vectors in R n . Hence, if differentiation corresponds to a linear transformation, it must be represented by a 
n x (n 4 1) matrix. For example, if p is an element of P-j —that is, 

2 

p(x) =ax 4 bx 4 c 


for some real numbers a, b, and c —then 

-j—p(x) = 2ax 4 b 
ax 

Evidently, if p(x) in P 2 corresponds to the vector ( a , b, c ) in Rp, then its derivative is in p^ and corresponds to the vector (2a, b) 
in p' 1 . Note that 

2 a 
_ b 

The operation differentiation, p)- pj _* p^, corresponds to a linear transformation p A - RR _* pf-, where 


♦ 


'2 0 O' 

a 

Jr, 

.0 1 °_ 

O 

C 


Some transformations from p n to p m do not correspond to linear transformations from R n ' 1 to For example, if we consider 

the transformation of ax 2 4 bx 4 c in Pj to |fl| in _p 0 , the space of all constants (viewed as polynomials of degree zero, plus the 
zero polynomial), then we find that there is no matrix that maps ( a , b, c) in p} to \a \ in R. Other transformations may correspond to 
transformations that are not quite linear, in the following sense. 


DEFINITION 


An affine transformation from R n to R m is a mapping of the form = T(ii) \ f, where T is a linear transformation from 
R n to R m and/is a (constant) vector in R m . 


The affine transformation S is a linear transformation if /is the zero vector. Otherwise, it isn't linear, because it doesn't satisfy 
Theorem 4.3.2. This may seem surprising because the form of S looks like a natural generalization of an equation describing a line, 
but linear transformations satisfy the Principle of Superposition 

T(ciu4c2v) =ciT(u) +c 2 T(v) 

for any scalars cj, c 2 and any vectors u, v in their domain. (This is just a restatement of Theorem 4.3.2.) Affine transformations 
with/nonzero don't have this property. 


Affine Transformations 


The mapping 


W = 


0 1 
1 0 


u 4 


1 

1 


is an affine transformation on p\ If u = (a, b), then 
























S( U) = 


0 1 

-1 0 


a 

b 


b + 1 

— ct j- 1 


1 

1 


The corresponding operation from p j to p^ takes ax 4- b to (h \ \)x — a I 1- 


The relationship between an action on p n and its corresponding action on the vector of coefficients in and the similarities 

between p n and will be explored in more detail later in this text. 

Interpolating Polynomials 

Consider the problem of interpolating a polynomial to a set of « + 1 points (xg, yg), ■ .., (x n , y n )- That is, we seek to find a curve 
p(x) = a m x m f •+ — I a^x \ ag of minimum degree that goes through each of these data points (Figure 4.4.2). Such a 

curve must satisfy 



Figure 4.4.2 

Interpolation 


yg=a m x™ +a m -xx™ 1 4-f^ix 0 +«o 

y 1 = <* m x™ + a m - ixj" -1 H-1- < 31*1 + ag 


y h — I a r. 


-1*™' 


-1 


H- \-a\x n ag 


Because the Xj are known, this leads to the following matrix system: 


1 x 0 *0 

1 xi x\ 

1 

1 X H X H 



Note that this is a square system when n = m- Taking n = m gives the following system for the coefficients of the interpolating 
polynomial p(x)'- 


*0 

*0 


2 

*1 

*1 

H —1 

4 - 


2 

X H 

*B 



(1) 


The matrix in 1 is known as a Vandermonde matrix ; column j is the second column raised element wise to the j — 1 power. The 



























linear system in 1 is said to be a Vandermonde system. 


Interpolating a Cubic 


To interpolate a polynomial to the data (-2, 11), (-1, 2), (1, 2), (2, -1), we form the Vandermonde system 1: 


1 *0 
1 XI 
1 *2 
1 


^0 A 0 


T A 1 


flo' 


>0 

fll 


71 

«2 


72 

a 3 


73 


For this data, we have 


'l 

-2 

4 

-8' 

'flo' 


if 

1 

- 1 

1 

- 1 

ai 


2 

1 

1 

1 

1 

a 2 


2 

1 

2 

4 

8 

a 3 


-1 


The solution, found by Gaussian elimination, is 


■flo' 


r 

ai 


1 

a 2 


1 

a 3 


-1 


and so the interpolant is p(x") = — x 3 4- x 2 4- x 4- 1. This is plotted in Figure 4.4.3, together with the data points, and we see that 
p(x") does indeed interpolate the data, as required. 



Figure 4.4.3 

The interpolant of Example 4 


♦ 


Newton Form 

The interpolating polynomial p(x) = a„x n 4- a H _i*” -1 1_|- a\x 4- arj is sa id to be written in its natural, or standard, form. But 

there is convenience in using other forms. For example, suppose we seek a cubic interpolant to the data yg), (jq, jq), (x 2 ,y 3 ) 

’ (*3,73)- Ifwe write 

3 2 

p{x)=a 3 x 4- a,'jx 4-aix 4- «o (21 


p(x) = a 3 (x -xn) 3 4- « 2 (x-xn) 2 4-ai(x-xn) 4-fln 


in the equivalent form 






















then the interpolation condition ^(xg) =70 immediately gives ag = 79 . This reduces the size of the system that must be solved 
from («4 1) x (« 4 1 ) to n x n- That is not much of a savings, but if we take this idea further, we may write 2 in the equivalent 
form 


p ( x ) = *3(4 - jrg )0 -*l )0 -*2) 4 -*o )0 -*l) 4 *i(x -xg) 4 *g 


( 3 ) 


which is called the Newton form of the interpolant. Set ^. — x,_i for j = ], 2, 3. The interpolation conditions give 

p Oo) = *o 
p (*i) =*1*1 + *0 

p {* 7 ) = *2(*1 + *2)*2 + *l(*l h*2) 3 *0 

^(^3) = *3(*1 4 *2 + *3)(*2 + *3)*3 4 *2(*1 b*2 + *3)(*2 4*3) -I *i(*i 4*2 +*3) + *0 

that is. 


'1 

0 


0 

0 


£■0 


>0' 

1 

*1 


0 

0 


*1 


7 1 

1 

h \ 4 - ^2 

(*1 

4 *2)*2 

0 


h 


72 

1 

^1 4 ^2 I - ^3 

(*14 *2 

1 *3) (*2 4 *3) 

(*1 4 *2 4 *3) (*2 

5 r j 

LO 

LO 

1_ 

h 


73 


Unlike the Vandermonde system 1, this system has a lower triangular coefficient matrix. This is a much simpler system. We may 
solve for the coefficients very easily and efficiently by forward-substitution, in analogy with back-substitution. In the case of 
equally spaced points arranged in increasing order, we have ^. = & > 0 , so 4 becomes 


'l 

0 

0 

0 ' 

£f) 


>0" 

1 

* 

0 

0 

*1 


71 

1 

2* 

2* 2 

0 

h 


72 

1 

3 * 

6* 2 

O". 

LO 

1_ 

h 


73 


Note that the determinant of 4 is nonzero exactly when ^. is nonzero for each i, so there exists a unique interpolant whenever the x* 
are distinct. Because the Vandermonde system computes a different form of the same interpolant, it too must have a unique 
solution exactly when the are distinct. 


Interpolating a Cubic in Newton Form 


To interpolate a polynomial in Newton form to the data (-2, 11), (-1, 2), (1, 2), (2, -1) of Example 5, we form the system 4: 


'l 

0 

0 

o ' 

*0 


'll' 

1 

1 

0 

0 

*1 


2 

1 

3 

6 

0 

*2 


2 

1 

4 

12 

12 

h 


1 


The solution, found by forward-substitution, is 

*0 = 
*0 1 *1= 
*g 4 - 3 *i 4 - 6*2 = 
*g 4 4 *i 4 - 12*2 4 12*3 = 


11 

2 *1 = -9 
2 *2= 3 

1 *3 = — 1 


and so, from 3, the interpolant is 

p(x)= — 1 - (x 4 2) (x 4 1) (x — 1) 4 3 - (x 4 2) (x 4 1) 4 ( — 9) - (x 4 2) 4 11 
= — (x 4 2)(x 4 1) (x — 1) 4 3(x 4 2) (x 4 1) — 9 (x 4 2) 4 11 


♦ 






















Converting between Forms 


The Newton form offers other advantages, but now we turn to the following question: If we have the coefficients of the 
interpolating polynomial in Newton form, what are the coefficients in the standard form? For example, if we know the coefficients 
in 

p(x) = b 3 (x-x 0 )(x-x\)(x-x 2 ) 4 b 2 (x -x 0 )(x -xi) 4 &i(x -x 0 ) + ^0 

because we have solved 4 in order to avoid having to solve the more complicated Vandermonde system 1, how can we get the 
coefficients in 2, 

3 2 

p(x)=a 3 x 4 a 2 x 4 a\x 4 
from Expanding the products in 3 gives 

p(x) =b 3 (x -x 0 )(x -xi)(x-x 2 ) 4 b 2 (x -x 0 )(x -x\) 4 &i(x -x 0 ) 4 &o 

= b 3 x 3 4 (b 2 - *3(^0 + *1 + x 2 ))x 2 

4 {b\-b 2 (x$ 4 xi) +b 3 {x$x\ V xqx 2 + *1*2))* 

4 &o — *0^1 +x$x\b 2 -x$x\x 2 b 3 


so 


a$ = bQ-x$b\ +x$x\b 2 -XQX\x 2 b 3 
a\ =b\ -b 2 {x$ 4xi) -\-b 3 {x 2 x\ 4 x 0 x 2 I-X 1 X 2 ) 
a 2 = b 2 -b 3 {xQ I- xi 4 x 2 ) 
a 3 = b 3 


This can be expressed as 


■flo’ 


1 x 0 

*0*1 

-X 0 X 1 X 2 

£■0 

fll 


0 1 

- (*0 + *l) 

*0* 1 4 X 0 X 2 4 X1X2 

*1 

«2 


0 0 

1 

- (x 0 4 xi 4x 2 ) 

h 

a 3 


0 0 

0 

1 

h 


(5) 


This is an important result! Solving the Vandermonde system 1 by Gaussian elimination would require us to form an nxn matrix 
that might have no nonzero entries and then to solve it using a number of arithmetic operations that grows in proportion to for 
large n. But solving the lower triangular system 4 requires an amount of work that grows in proportion to yp for large n, and using 
5 to compute the coefficients ag, a\, a 2 , a 3 also requires an amount of work that grows in proportion to yp for large n. Hence, for 
large n, the latter approach is an order of magnitude more efficient. The two-step procedure of solving 4 and then using the linear 
transformation 5 is a superior approach to solving 1 when n is large (Figure 4.4.4). 


L.tnLMr Frans t or imEion 



pix) in New Lon form 


pi u in Sliiiidiird form 




Figure 4.4.4 

Indirect route to conversion from Newton form to standard form 


Changing Forms 


In Example 4 we found that fl g = 1 , a \ = 1, a 2 = 1> a 3 = — 1> whereas in Example 5 we found that &g = 11, & j = — 9,b 2 = 3, 
= _ 1 for the same data. From 5, with *g = _ 2- xj = — 1, xj = T we expect that 














which checks. 


1 


1 


1 


-1 



1 

0 

0 

0 


2 

1 

0 

0 


2 

3 

1 

0 


-2 

-1 

2 

1 


11 

-9 

3 

- 1 


There is another approach to solving 1, based on the Fast Fourier Transform, that also requires an amount of work proportional to 
yp-. The point for now is to see that the use of linear transformations on P”" 1-1 can help us perform computations involving 
polynomials. The original problem—to fit a polynomial of minimum degree to a set of data points—was not couched in the 
language of linear algebra at all. But rephrasing it in those terms and using matrices and the notation of linear transformations on 
has allowed us to see when a unique solution must exist, how to compute it efficiently, and how to transform it among 
various forms. 


Exercise Set 4.4 



Click here for Just Ask! 


Identify the operations on polynomials that correspond to the following operations on vectors. Give the resulting polynomial. 

1. 


r 


'3' 

2 

-2 

0 

-i 


2 


(b) 

'4' 


V 

5 

3 

+ 6 

2 


0 


1 


f 


o" 

2 


2 

1 

— 

0 

-2 


-2 

1 


u 


(d) 


A 


4 

-3 

7 

1 


2 . 


(a) Consider the operation on _p 2 that takes a x^ 4- bx 4- c to C x A \ bx \ a- Does it correspond to a linear transformation 
from to ^3? If so, what is its matrix? 























(b) Consider the operation on p^, that takes ax? 4 - hr? 4 cx + d to c?r' — hr?" — ax \ d■ Does it correspond to a linear 
transformation from p} to p?l If so, what is its matrix? 


3. 

(a) Consider the transformation of ax 2 4 - hx c in p^ to |a| in p^. Show that it does not correspond to a linear 
transformation by showing that there is no matrix that maps (a, b, c) in B? to M in R. 

(b) Does the transformation of ax 1 | bx I c in p 2 to a in py, correspond to a linear transformation from p? to R! 


(a) Consider the operation P2 _» P 3 that takes p(x) in p 2 to xp(x) in p- s - Does this correspond to a linear 

transformation from p} to If so, what is its matrix? 

(b) Consider the operation pf- P 2 _> p- s that takes p(x) in p n to (x — l)p(x) in P n+ \- Does this correspond to a linear 

transformation from p} Xo p^'l If so, what is its matrix? 

(c) Consider the operation w. f'2 _ * P3 that takes p(x) in p n to xp(x') [ 1 in P n+ \ - Does this correspond to a linear 

transformation from P;-' to P; 4 ? If so, what is its matrix? 


5. (For Readers Who Have Studied Calculus) What matrix corresponds to differentiation in each case? 


(a) D: P 3 - 

^P2 

(b) D: P 4 - 

-P 3 

(c) D: P 5 - 

-Pa 


6. (For Readers Who Have Studied Calculus) What matrix corresponds to differentiation in each case, assuming we represent 
p(x)=a n x n I H- \-a\x-\ ao as the vector a \,a n _\, a H )? 

Note This is the opposite of the ordering of coefficients we have been using. 


(a) D: P 3 —♦ P 2 

(b) D: P 4 —» P 3 

(c) D: P 5 —* P 4 



Consider the following matrices. What is the corresponding transformation on polynomials? Indicate the domain p i and the 
7* codomain Pj. 


(a) 


1 1 
1 - 1 


(b) 


1 0 

1 1 

2 - 1 


(c) 


10 2-1 
2 11 3 


(d) 


0 0 0 
0 1 0 
0 0 0 


(e) [0 1 0] 


Consider the space of all functions of the form a-\ b cos(x) 4- c sin Or) where a, b, c are scalars. 

8 . 


(a) What matrix, if any, corresponds to the change of variables x —> x — ^ / 2> assuming that we represent a function in 
this space as the vector (a,b,c)7 


(b) What matrix corresponds to differentiation of functions on this space? 


Consider the space of all functions of the form a-\ bt-\ ce f -\- de f , where a, b, c, d are scalars. 

9. 

(a) What function in the space corresponds to the sum of (1, 2, 3, 4) and (-1, -2, 0, -1), assuming that we represent a 
function in this space as the vector b, c, d) 1 - 


(b) Is cosh(£) in this space? That is, does cosh(^) correspond to some choice of a, b, c, dl 


(c) What matrix corresponds to differentiation of functions on this space? 


Show that the Principle of Superposition is equivalent to Theorem 4.3.2. 


10 . 







Show that an affine transformation with/nonzero is not a linear transformation. 


11 . 

Find a quadratic interpolant to the data (-1, 2), (0, 0), (1, 2) using the Vandermonde system approach. 

12 . 


13. 

(a) Find a quadratic interpolant to the data (-2, 1), (0, 1), (1, 4) using the Vandermonde system approach from 1. 

(b) Repeat using the Newton approach from 4. 

14. 

(a) Find a polynomial interpolant to the data (-1, 0), (0, 0), (1, 0), (2, 6) using the Vandermonde system approach from 1. 

(b) Repeat using the Newton approach from 4. 

(c) Use 5 to get your answer in part (a) from your answer in part (b). 

(d) Use 5 to get your answer in part (b) from your answer in part (a) by finding the inverse of the matrix. 

(e) What happens if you change the data to (-1, 0), (0, 0), (1, 0), (2, 0)? 


15. 


16 . 


(a) Find a polynomial interpolant to the data (-2, -10), (-1, 2), (1, 2), (2, 14) using the Vandermonde system approach 
from 1. 


(b) Repeat using the Newton approach from 4. 


(c) Use 5 to get your answer in part (a) from your answer in part (b). 


(d) Use 5 to get your answer in part (b) from your answer in part (a) by finding the inverse of the matrix. 


Show that the determinant of the 2x2 Vandermonde matrix 


1 a 
1 b 

can be written as (b — a) and that the determinant of the 3x3 Vandermonde matrix 


det 


1 a a 1 
1 b b 2 


1 c c* 

can be written as (b — a)(c — a)(c — b)- Conclude that a unique straight line can be fit through any two points (xq, Vo)’ 



(x\, yi") with ?;q and jq distinct, and that a unique parabola (which may be degenerate, such as a line) can be fit through any 
three points (x 0 ,;/ 0 ), (^1.7l)’ 02.72) with x 0 , ;q, and x 2 distinct. 


17. 

(a) What form does 5 take for lines? 


(b) What form does 5 take for quadratics? 


(c) What form does 5 take for quartics? 


Discussion 

Discovery 


18. (For Readers Who Have Studied Calculus) 


(a) Does indefinite integration of functions in correspond to some linear transformation from 

tO £H+2? 

(b) Does definite integration (from * = 0 to * = 1) of functions in p n correspond to some linear 
transformation from J?”" 1 " 1 to /?? 


19. (For Readers Who Have Studied Calculus) 


(a) What matrix corresponds to second differentiation of functions from p 2 (giving functions in 

P 0 ) ? 

(b) What matrix corresponds to second differentiation of functions from p 3 (giving functions in 
Pt)? 


(c) Is the matrix for second differentiation the square of the matrix for (first) differentiation? 


Consider the transformation from Pj to _p 2 associated with the matrix 

'0 0 O' 

0 1 0 
0 0 0 

and the transformation from t0 Pq associated with the matrix 

[0 1 0 ] 

These differ only in their codomains. Comment on this difference. In what ways (if any) is it 
important? 

The third major technique for polynomial interpolation is interpolation using Lagrange 
interpolating polynomials. Given a set of distinct x-values xq, x j, ... x n , define the n + 1 Lagrange 
interpolating polynomials for these values by (for j = Q, 1, ... n ) 






L r„v_ 

‘ J! 0] “*0)(*I -*lM>] “*l-l)0] -*l+l)-(*I -*h) 

Note that £,•(*) is a polynomial of exact degree n and that Lj(xj) = 0 if i * j, and Lj(x{) = 1- It 
follows that we can write the polynomial interpolant to (xq, ^ q ), .. (x y n ) in the form 

p(x) = cgZnOO + cili(x) H- \-c„L„{x) 

whereCj= 7 j,j = 0, 1, 


(a) Verify that p ( x ) =y 0 L 0 (x) f y (x) + - 4- y n L n (x) is the unique interpolating 
polynomial for this data. 


(b) What is the linear system for the coefficients eg, cj, c n , corresponding to 1 for the 
Vandermonde approach and to 4 for the Newton approach? 

(c) Compare the three approaches to polynomial interpolation that we have seen. Which is most 
efficient with respect to finding the coefficients? Which is most efficient with respect to 
evaluating the interpolant somewhere between data points? 


Generalize the result in Problem 16 by finding a formula for the determinant of an nxn 
22. Vandermonde matrix for arbitrary n. 


23. 


The norm of a linear transformation Ta R n —> R n can be defined by 


II711 £ = max 


IITOll 

INI 


where the maximum is taken over all nonzero x in R n . (The subscript indicates that the norm of the 
linear transformation on the left is found using the Euclidean vector norm on the right.) It is a fact 
that the largest value is always achieved—that is, there is always some xq in R n such that 
imi* = rnax(||T(in) || /1|xo||)- What are the norms of the linear transformations Tj\ with the 
following matrices? 


(a) 


2 0 
0 1 


(b) 


1 0 
0 - 1 


(c) 


2 0 

0 -3 


(d) 


1/^2 

1/^2 


1/^2 

- 1/^2 
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Chapter 4 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Section 4.1 


Tl. (Vector Operations in R n ) With most technology utilities, the commands for operating on vectors in R n are the same as 
those for operating on vectors in and p\ and the command for computing a dot product produces the Euclidean inner 
product in R }1 . Use your utility to perform computations in Exercises 1, 3, and 9 of Section 4.1. 


Section 4.2 

Tl. (Rotations) Find the standard matrix for the linear operator on r} that performs a counterclockwise rotation of 45° about the 
x-axis, followed by a counterclockwise rotation of 60° about the y-axis, followed by a counterclockwise rotation of 30° about 
the z-axis. Then find the image of the point (1, 1, 1) under this operator. 


Section 4.3 

Tl. (Projections) Use your utility to perform the computations for Q = ^ / 6 in Example 6. Then project the vectors (1,1) and (1, 
-5). Repeat for Q = K / 4 , n / 3 , n / 2 , 


Section 4.4 

Tl. (Interpolation) Most technology utilities have a command that performs polynomial interpolation. Read your 

documentation, and find the command or commands for fitting a polynomial interpolant to given data. Then use it (or them) 
to confirm the result of Example 5. 
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CHAPTER 


General Vector Spaces 


NTRODUCTION In the last chapter we generalized vectors from 2- and 3-space to vectors in n-space. In this chapter 
we shall generalize the concept of vector still further. We shall state a set of axioms that, if satisfied by a class of objects, will 
entitle those objects to be called “vectors.” These generalized vectors will include, among other things, various kinds of 
matrices and functions. Our work in this chapter is not an idle exercise in theoretical mathematics; it will provide a powerful 
tool for extending our geometric visualization to a wide variety of important mathematical problems where geometric intuition 
would not otherwise be available. We can visualize vectors in g 2 and g 3 as arrows, which enables us to draw or form mental 
pictures to help solve problems. Because the axioms we give to define our new kinds of vectors will be based on properties of 
vectors in g 2 and g 3 , the new vectors will have many familiar properties. Consequently, when we want to solve a problem 
involving our new kinds of vectors, say matrices or functions, we may be able to get a foothold on the problem by visualizing 
what the corresponding problem would be like in g 2 and g 3 . 
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5 ^ In this section we shall extend the concept of a vector by extracting the 

■ • most important properties of familiar vectors and turning them into axioms. 

REAL VECTOR SPACES Thus, when a set of objects satisfies these axioms, they will automatically 

have the most important properties of familiar vectors, thereby making it 
reasonable to regard these objects as new kinds of vectors. 


Vector Space Axioms 

The following definition consists of ten axioms. As you read each axiom, keep in mind that you have already seen each of 
them as parts of various definitions and theorems in the preceding two chapters (for instance, see Theorem 4.1.1). 
Remember, too, that you do not prove axioms; they are simply the “rules of the game.” 


DEFINITION 


Let V be an arbitrary nonempty set of objects on which two operations are defined: addition, and multiplication by scalars 
(numbers). By addition we mean a rule for associating with each pair of objects u and v in V an object u |- v , called the 
sum of u and v; by scalar multiplication we mean a rule for associating with each scalar k and each object u in V an 
object £u, called the scalar multiple of u by k. If the following axioms are satisfied by all objects u, v , w in V and all 
scalars k and m, then we call V a vector space and we call the objects in V vectors. 

If u and v are objects in V, then u + v is in y 

2 . u + v = v I- u 

3. u -f (v 4 w) = (u 4 v) 4 w 

There is an object 0 in y, called a zero vector for y, such that 0 4 - u = u 4- 0 = u for all u in y. 

For each u in y, there is an object _ u in y, called a negative of u , such that u | ( — u) = ( — u) | u = 0- 

6. If k is any scalar and u is any object in y, then £u is in y. 

7. Ar(n 4 v) = £11 4 - jfcv 

8. (k 4- ra)u = bi+ m\i 

t(™i) — (km) (u) 








lu = u 


Remark Depending on the application, scalars may be real numbers or complex numbers. Vector spaces in which the 
scalars are complex numbers are called complex vector spaces, and those in which the scalars must be real are called real 
vector spaces. In Chapter 10 we shall discuss complex vector spaces; until then, all of our scalars will be real numbers. 


The reader should keep in mind that the definition of a vector space specifies neither the nature of the vectors nor the 
operations. Any kind of object can be a vector, and the operations of addition and scalar multiplication may not have any 
relationship or similarity to the standard vector operations on Bf. The only requirement is that the ten vector space axioms 
be satisfied. Some authors use the notations © and for vector addition and scalar multiplication to distinguish these 
operations from addition and multiplication of real numbers; we will not use this convention, however. 

Examples of Vector Spaces 

The following examples will illustrate the variety of possible vector spaces. In each example we will specify a nonempty set 
V 

and two operations, addition and scalar multiplication; then we shall verify that the ten vector space axioms are satisfied, 
thereby entitling V, with the specified operations, to be called a vector space. 


R n Is a Vector Space 


The set V = pf with the standard operations of addition and scalar multiplication defined in Section 4.1 is a vector space. 
Axioms 1 and 6 follow from the definitions of the standard operations on pf\ the remaining axioms follow from Theorem 


The three most important special cases of R n are R (the real numbers), p} (the vectors in the plane), and p} (the vectors in 
3-space). 


A Vector Space of 2 x 2 Matrices 


Show that the set V of all 2 x 2 matrices with real entries is a vector space if addition is defined to be matrix addition and 
scalar multiplication is defined to be matrix scalar multiplication. 


Solution 


In this example we will find it convenient to verify the axioms in the following order: 1, 6, 2, 3, 7, 8, 9, 4, 5, and 10. Let 


“11 «12 
U 21 u 22 


and 


Vll V12 
V 21 v 22 


To prove Axiom 1, we must show that u _|_ v is an object in V; that is, we must show that u | v is a 2 x 2 matrix. But this 
follows from the definition of matrix addition, since 











Similarly, Axiom 6 holds because for any real number k, we have 

ku\\ ku 12 
hi2\ hi22 

so ,tu is a 2 x 2 matrix and consequently is an object in V. 


£u = k 


“11 “12 
“21 “22 


Axiom 2 follows from Theorem 1.4.1a since 


'“11 

“12' 

+ 

'Vll 

V12' 

“21 

“22 

V 21 

v 22 


>11 

V 12 ~ 

+ 

'“11 

“ 12 ' 

V 21 

V 22 

“21 

“22 


Similarly, Axiom 3 follows from part ( b ) of that theorem; and Axioms 7, 8, and 9 follow from parts (/?), (j), and (/), 
respectively. 


To prove Axiom 4, we must find an object 0 in V such that 0 + u = u + 0 = u for all u in V. This can be done by defining 0 
to be 


With this definition, 


0 + u = 


0 

0 


+ 


“11 

“21 


“12 

“22 


“11 “12 
“21 “22 


and similarly u + 0 = u- To prove Axiom 5, we must show that each object u in V has a negative _ u such that 
u + ( — u) = 0 and ( — u) + u = 0- This can be done by defining the negative of u to be 


-“11 -“12 
-“21 -“22 


With this definition, 


“11 

“ 12 ' 

+ 

' -“11 

-“ 12 ' 

= 

'0 

o ' 

“21 

“22 

_ -“21 

-“22 


_0 

0 _ 


and similarly ( _ u ) + u = 0 Finally, Axiom 10 is a simple computation: 


'“11 

“ 12 ' 


'“11 

“ 12 ' 

“21 

“22 


“21 

“22 


♦ 


A Vector Space of mxn Matrices 


Example 2 is a special case of a more general class of vector spaces. The arguments in that example can be adapted to show 
that the set V of all m x n matrices with real entries, together with the operations of matrix addition and scalar multiplication, 
is a vector space. The mxn zero matrix is the zero vector 0, and if u is the mxn matrix U, then the matrix _ JJ is the 
negative _ u of the vector We shall denote this vector space by the Symbol M mn - 


♦ 


A Vector Space of Real-Valued Functions 


Let The the set of real-valued functions defined on the entire real line ( _ oo , oc. )• If f = / (x) and s = g(x) are two such 












































functions and k is any real number, define the sum function f + g and the scalar multiple £f, respectively, by 

(f + g) (x) =./ 00 + #00 and (kf)(x)=k.f(x) 

In other words, the value of the function f .p g at x is obtained by adding together the values of f and g at x (Figure 5.1.1a). 
Similarly, the value of if at x is k times the value of f at x (Figure 5.1.1b). In the exercises we shall ask you to show that V is 
a vector space with respect to these operations. This vector space is denoted byi?(_ <x>, do) -Iff and g are vectors in this 
space, then to say that f = g is equivalent to saying that f (x) — g(x) for all x in the interval ( _ oo , oo) • 





Figure 5.1.1 


The vector Q in F ( ^ M , oo ) is the constant function that is identically zero for all values of x. The graph of this function is 
the line that coincides with the x-axis. The negative of a vector/is the function _ f — _ j (x). Geometrically, the graph of 
— f is the reflection of the graph of f across the x-axis (Figure 5.1.1c). 

♦ 


Remark In the preceding example we focused on the interval ( _ oo , oo )• Had we restricted our attention to some closed 
interval or some open interval (a, b), the functions defined on those intervals with the operations stated in the 

example would also have produced vector spaces. Those vector spaces are denoted by F[a,b] and F{a, b), respectively. 


Let V = F? and define addition and scalar multiplication operations as follows: If u —{u\, ui) and y — ( V i, v-/b then define 

EXamklE 5 A Set That Is Not a Vector . 

uT v = (ui -F vi, U2 T vi) 















and if k is any real number, then define 


£u = (£wi, 0) 

For example, if u = (2, 4), v = ( - 3, 5), and k = 7, then 

u + v=(2+(-3),4 + 5) = (-l,9) 

£u = 7u = (7 - 2, 0) = (14, 0) 

The addition operation is the standard addition operation on R 2 , but the scalar multiplication operation is not the standard 
scalar multiplication. In the exercises we will ask you to show that the first nine vector space axioms are satisfied; however, 
there are values of u for which Axiom 10 fails to hold. For example, if u = (u\, ui) i s such that y 2 * 0> then 

lu= 1 (wi, ui) = (1 -u\, 0) = («i, 0) 

Thus V is not a vector space with the stated operations. 


Every Plane through the Origin Is a Vector Space 


Let V be any plane through the origin in R-'. We shall show that the points in V form a vector space under the standard 
addition and scalar multiplication operations for vectors in r}. From Example 1, we know that p} itself is a vector space 
under these operations. Thus Axioms 2, 3, 7, 8 , 9, and 10 hold for all points in r} and consequently for all points in the 
plane V. We therefore need only show that Axioms 1, 4, 5, and 6 are satisfied. 

Since the plane V passes through the origin, it has an equation of the form 

ax -\- by \ cz— 0 


(Theorem 3.5.1). Thus, if u = («j, U 2 , ui) and v = (vj, V 2 , V 3 ) are points in V, then au\ 4 bu 2 4 cu-i = 0 and 
av 1 +- bv 2 4 - CV 3 = 0- Additing these equations gives 


a(u 1 T vi) 4- b(u 2 4- v?) 4- c(u 3 4- V 3 ) = 0 
This equality tells us that the coordinates of the point 


u + v = (u 1 4- vi, U 2 4- V 2 , U 3 4- V 3 ) 

satisfy 1; thus u _| v lies in the plane V. This proves that Axiom 1 is satisfied. The verifications of Axioms 4 and 6 are left as 
exercises; however, we shall prove that Axiom 5 is satisfied. Multiplying au ^ _|_ £ U2 4- cu -2 = 0 through by _ l gives 

a( — u 1 ) 4*&( — U 2 ) 4- c( — U 3 ) — 0 
Thus _ u = ( _ u\, — ui, — ui) hes in V. This establishes Axiom 5. 


♦ 


The Zero Vector Space 


Let V consist of a single object, which we denote by 0, and define 

0 4-0=0 and ; 4 ) = 0 

for all scalars k. It is easy to check that all the vector space axioms are satisfied. We call this the zero vector space. 

♦ 


Some Properties of Vectors 

As we progress, we shall add more examples of vector spaces to our list. We conclude this section with a theorem that gives 
a useful list of vector properties. 






THEOREM 5.1.1 


Let V be a vector space, u a vector in V, and k a scalar; then: 

(a) Ou = 0 

(b) *o = o 

(c) (-l)u= -u 

(d) Ifkii - 0, then k = 0 or u = 0- 

We shall prove parts (a) and (c) and leave proofs of the remaining parts as exercises. 


'if (a) We can write 



Ou 4- [Ou + ( - Ou) ] = Ou + ( - Ou) 
Ou -I- 0 = 0 

0u = 0 


[Axiom 5] 
[Axiom 5 ] 
[Axiom 4] 


if (c) To show that ( _ l) u = — u, we must demonstrate that u + ( — l)u = 0- To see this, observe that 

u-\- ( — l)u = lu 4- ( — l)w [AxiomlO] 

= (l + (-.l))a [Axiom S] 


= 0u 
= 0 


[Property of numbers] 
[Part (a) above] 


Exercise Set 5.1 


Click here for Just Ask! 






In Exercises 1-16 a set of objects is given, together with operations of addition and scalar multiplication. Determine which 
sets are vector spaces under the given operations. For those that are not vector spaces, list all axioms that fail to hold. 

The set of all triples of real numbers (x, y, z) with the operations 
1 ‘ 0 , y, z) I (>■', y, z') = (x h r'jl y ! , z 4 z ! ) and k(x, y, z) = {kx, y, z) 

The set of all triples of real numbers (x, y, z) with the operations 
2 * x, y, z ) + (x', y, z ! ) = (x 4 x', v 4 y f , z I z ! ) and k{x, y, z) = ( 0 , 0 , 0 ) 

The set of all pairs of real numbers (x, y) with the operations 
3 " 0, 7) 4 (x\ y') = (x 4 x 1 ', y 4 y 1 ) and £(x, 7 ) = (2kx, 2ky) 

The set of all real numbers x with the standard operations of addition and multiplication. 

4. 


The set of all pairs of real numbers of the form (x, 0) with the standard operations on p}. 


6 . 


The set of all pairs of real numbers of the form (x, y), where x > 0, with the standard operations on 


The set of all n-tuples of real numbers of the form (x, x,..., x) with the standard operations on R n . 

7. 

The set of all pairs of real numbers (x, 7 ) with the operations 
8 " (x,.y) 4 (x',y) = (x 4 x ! 4 \,y f y ! 4 1 ) and k(x, y) = (kx, ky) 

The set of all 2 x 2 matrices of the form 

9 - ' r, 11 

_i t_ 

with the standard matrix addition and scalar multiplication. 

The set of all 2 x 2 matrices of the form 

10 . ' r n1 

a 0 

_° b_ 

with the standard matrix addition and scalar multiplication. 

The set of all real-valued functions/defined everywhere on the real line and such that / (l) = 0, with the operations 
11* defined in Example 4. 

The set of all 2 x 2 matrices of the form 
a a 4 b 
a I b b 

with matrix addition and scalar multiplication. 


13 . 


The set of all pairs of real numbers of the form (1, x) with the operations 



The set of polynomials of the form a 4 bx with the operations 

(c 0 h a\x) 4 (&o + ^1*) = ( fl 0 + ^o) I ( fl l I £i)x and 4 a\x) — (£ao) -I (£ai)x 

The set of all positive real numbers with the operations 

15. , , k 

x 4 - y — xy and kx — x 

The set of all pairs of real numbers (*, y) with the operations 
16 ’ O, y) I (x f , y ! ) = (xx ! , yy 1 ) and k(x, y) = (kx, ky) 

Show that the following sets with the given operations fail to be vector spaces by identifying 

17. 

(a) The set of all triples of real numbers with the standard vector addition but with scalar 
*0, y, Z) = (k 2 x, k 2 y, k 2 z). 


all axioms that fail to hold, 
multiplication defined by 


(b) The set of all triples of real numbers with addition defined by y ; z ) 4 - (u, v, w) = (z |-vf,y 4 v,iH u) and 
standard scalar multiplication. 


(c) The set of all 2 x 2 invertible matrices with the standard matrix addition and scalar multiplication. 


Show that the set of all 2 x 2 matrices of the form 
and scalar multiplication defined by k 


a 1 


_1 b_ 



a 1 
1 b 
ka 1 
1 kb 


with addition defined by 


£ 3 ! 1 

1 b 


+ 


C 1 

1 d 


ale 1 
1 b + d 


is a vector space. What is the zero vector in this space? 


19. 


(a) Show that the set of all points in g? lying on a line is a vector space, with respect to the standard operations of 
vector addition and scalar multiplication, exactly when the line passes through the origin. 


(b) Show that the set of all points in p} lying on a plane is a vector space, with respect to the standard operations of 
vector addition and scalar multiplication, exactly when the plane passes through the origin. 


Consider the set of all 2 x 2 invertible matrices with vector addition defined to be matrix multiplication and the standard 
20. scalar multiplication. Is this a vector space? 

Show that the first nine vector space axioms are satisfied if y — j ? 2 has the addition and scalar multiplication operations 
21- defined in Example 5. 


Prove that a line passing through the origin in g? is a vector space under the standard operations on g-'. 


22 . 




Complete the unfinished details of Example 4. 


23. 

Complete the unfinished details of Example 6. 

24. 


Discussion 

Discovery 


We showed in Example 6 that every plane in gl that passes through the origin is a vector 
25* space under the standard operations on j? 3 . Is the same true for planes that do not pass through 
the origin? Explain your reasoning. 


It was shown in Exercise 14 above that the set of polynomials of degree 1 or less is a vector 
26. space under the operations stated in that exercise. Is the set of polynomials whose degree is 
exactly 1 a vector space under those operations? Explain your reasoning. 


Consider the set whose only element is the moon. Is this set a vector space under the 
27. operations moon + moon = moon and k(moon)=moon for every real number kl Exaplain your 
reasoning. 


Do you think that it is possible to have a vector space with exactly two distinct vectors in it? 
28. Explain your reasoning. 


The following is a proof of part ( b ) of Theorem 5.1.1. Justify each step by filling in the blank 
29. line with the word hypothesis or by specifying the number of one of the vector space axioms 
given in this section. 

Hypothesis: Let u be any vector in a vector space V, 0 the zero vector in V, and k a scalar. 

Conclusion: Then £0=0- 

Proof: 


1. First, £0 E An = A;(0 + u)-_ 

2. = Am- 

3. Since An is in V, _ An is in V. _ 

4. Therefore, (AO f An) 4- ( — An) = An I ( — An)- 

5- AO 4- (An E ( — An)) = An f ( — An)_ 


AO 4-0 = 0 


6 . 










7. Finally, 


£0 = 0 


Prove part ( d ) of Theorem 5.1.1. 

30 . 

The following is a proof that the cancellation law for addition holds in a vector space. Justify 
31* each step by filling in the blank line with the word hypothesis or by specifying the number of 
one of the vector space axioms given in this section. 

Hypothesis: Let u, v, and w be vectors in a vector space V and suppose that u 4 w = V f w 
Conclusion: Then u — y . 

Proof: 

1. First, ( u 4 - w ) 4 - ( - w ) and ( v 4 - w) 4 - ( - w) are vectors in V. _ 

2. Then ( u 4 - w ) 4- ( - w) = (v T w) 4 - ( — w) •- 

3. The left side of the equality in step (2) is ( u 4 - w) | ( — w) =u+ (w 4 - ( — w)) 


= 11- 

4. The right side of the equality in step (2) is ( v 4 w) | ( — w) = v 4 - (w 4 - ( — w)) 


= v- 

From the equality in step (2), it follows from steps (3) and (4) that u — v . 

Do you think it is possible for a vector space to have two different zero vectors? That is, is it 

32. possible to have two different vectors 0 j and 0 j such that these vectors both satisfy Axiom 4? 
Explain your reasoning. 

Do you think that it is possible for a vector u in a vector space to have two different 

33. negatives? That is, is it possible to have two different vectors ( — u) j and ( _ u ) 2 , both of 
which satisfy Axiom 5? Explain your reasoning. 

The set of ten axioms of a vector space is not an independent set because Axiom 2 can be 

34. deduced from other axioms in the set. Using the expression 

(u 4- v) — (v 4- u) 

and Axiom 7 as a starting point, prove that u + v = v I u- 

Hint You can use Theorem 5.1.1 since the proof of each part of that theorem does not use 
Axiom 2. 
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5.2 

SUBSPACES 


It is possible for one vector space to be contained within another vector 
space. For example, we showed in the preceding section that planes 
through the origin are vector spaces that are contained in the vector space 
R 3 . In this section we shall study this important concept in detail. 


A subset of a vector space V that is itself a vector space with respect to the operations of vector addition and scalar 
multiplication defined on V is given a special name. 


DEFINITION 


A subset W of a vector space V is called a subspace of V if W is itself a vector space under the addition and scalar 
multiplication defined on V. 


In general, one must verify the ten vector space axioms to show that a set W with addition and scalar multiplication forms a 
vector space. However, if W is part of a larger set V that is already known to be a vector space, then certain axioms need not 
be verified for W because they are “inherited” from V. For example, there is no need to check that u + v = v | u (Axiom 2) 
for W because this holds for all vectors in V and consequently for all vectors in W. Other axioms inherited by W from V are 3, 
7, 8, 9, and 10. Thus, to show that a set Vkis a subspace of a vector space V, we need only verify Axioms 1, 4, 5, and 6. The 
following theorem shows that even Axioms 4 and 5 can be omitted. 


THEOREM 5.2.1 


If W is a set of one or more vectors from a vector space V, then W is a subspace ofV if and only if the following 
conditions hold. 

(a) If u and v are vectors in W, then u |- v is in W. 

(b) If k is any scalar and u is any vector in W, then £u is in W. 


Proof If W is a subspace of V, then all the vector space axioms are satisfied; in particular, Axioms 1 and 6 hold. But these 
are precisely conditions (a) and ( b ). 

Conversely, assume conditions (a) and ( b ) hold. Since these conditions are vector space Axioms 1 and 6, we need only show 
that W satisfies the remaining eight axioms. Axioms 2, 3, 7, 8, 9, and 10 are automatically satisfied by the vectors in W since 
they are satisfied by all vectors in V. Therefore, to complete the proof, we need only verify that Axioms 4 and 5 are satisfied 
by vectors in W. 

Let u be any vector in W. By condition (b), £u is in W for every scalar k. Setting k = Q, it follows from Theorem 5.1.1 that 
0u = 0 is in W, and setting k = — 1, it follows that ( _ l) u = _ u is in W. 










Remark A set W of one or more vectors from a vector space V is said to be closed under addition if condition (a) in 
Theorem 5.2.1 holds and closed under scalar multiplication if condition (b) holds. Thus Theorem 5.2.1 states that W is a 
subspace ofV if and only ifW is closed under addition and closed under scalar multiplication. 


Testing for a Subspace 


In Example 6 of Section 5.1 we verified the ten vector space axioms to show that the points in a plane through the origin of 
pf form a subspace of pf. In light of Theorem 5.2.1 we can see that much of that work was unnecessary; it would have been 
sufficient to verify that the plane is closed under addition and scalar multiplication (Axioms 1 and 6 ). In Section 5.1 we 
verified those two axioms algebraically; however, they can also be proved geometrically as follows: Let IT be any plane 
through the origin, and let u and v be any vectors in IT. Then u 4 . v must lie in IT because it is the diagonal of the 
parallelogram determined by u and v (Figure 5.2.1), and must lie in IT for any scalar k because £u lies on a line through u . 
Thus IT is closed under addition and scalar multiplication, so it is a subspace of p^. 



Figure 5.2.1 

The vectors u 4 . v and kn both lie in the same plane as u and v . 


♦ 


Lines through the Origin Are Subspaces 


Show that a line through the origin of p 3 is a subspace of pf. 


Solution 

Let IT be a line through the origin of pf. It is evident geometrically that the sum of two vectors on this line also lies on the 
line and that a scalar multiple of a vector on the line is on the line as well (Figure 5.2.2). Thus IT is closed under addition and 
scalar multiplication, so it is a subspace of pf. In the exercises we will ask you to prove this result algebraically using 
parametric equations for the line. 








y~ 

/ 

(fj) U r is closed under addition, 




w 


(/>) U- J i s closed under scalar 
multiplication. 

Figure 5.2.2 

♦ 


Subset of R 2 That Is Not a Subspace 


Let W be the set of all points (x, y ) in F? such that * > Q and y > Q. These are the points in the first quadrant. The set W is 
not a subspace of gZ since it is not closed under scalar multiplication. For example, v = (1, 1) lies in W, but its negative 
( _ l)v= — v = ( — 1, —1) does not (Figure 5.2.3). 



Figure 5.2.3 

W is not closed under scalar multiplication. 




Every nonzero vector space V has at least two subspaces: V itself is a subspace, and the set {0} consisting of just the zero 
vector in V is a subspace called the zero subspace. Combining this with Examples Example 1 and Example 2, we obtain the 
following list of subspaces of g 2 and p}\ 










Subspaces of g 2 


Subspaces of g? 


{ 0 } 


{ 0 } 


Lines through the origin Lines through the origin 


R 2 


Planes through the origin 



Later, we will show that these are the only subspaces of g2 and g\ 


Subspaces of M nn 


From Theorem 1.7.2, the sum of two symmetric matrices is symmetric, and a scalar multiple of a symmetric matrix is 
symmetric. Thus the set of n x n symmetric matrices is a subspace of the vector space M nyi of all nxn matrices. Similarly, 
the set of n x n upper triangular matrices, the set of n x n lower triangular matrices, and the set of n x n diagonal matrices all 
form subspaces of M nn , since each of these sets is closed under addition and scalar multiplication. 

♦ 


A Subspace of Polynomials of Degree < n 


Let n be a nonnegative integer, and let W consist of all functions expressible in the form 

j P(x)=l3q-| a\x 4-b a„x n 


( 1 ) 


where a q, a n are real numbers. Thus W consists of all real polynomials of degree n or less. The set W is a subspace of the 
vector space of all real-valued functions discussed in Example 4 of the preceding section. To see this, let p and q be the 
polynomials 

p(x) — ao a\x H- \-a„x n and q(x) — 4-H-1 b n x n 


Then 


(P 4 Q)00 — P( x ) T <Z( X ) - C«0 + ^o) + (fli + Ai)xH -h (a n + b n )x” 


and 


= bp(x) — (^o) + (ka\)x A -h (ka n )x n 


These functions have the form given in 1, so p T q and £p lie in IT. As in Section 4.4, we shall denote the vector space IT in 
this example by the symbol j p H . 









The CMYK Color Model 


Color magazines and books are printed using what is called a CMYK color model. Colors in this model are created 
using four colored inks: (C), (M), (Y), and (K). The colors can be created either by mixing inks of the four types and 
printing with the mixed inks (the spot color method) or by printing dot patterns (called rosettes ) with the four colors and 
allowing the reader's eye and perception process to create the desired color combination (the process color method). 
There is a numbering system for commercial inks, called the Pantone Matching System, that assigns every commercial 
ink color a number in accordance with its percentages of cyan, magenta, yellows, and black. Oneway to represent a 
Pantone color is by associating the four base colors with the vectors 


c = (l,0, 0, 0) 
m= (0,1,0, 0) 
y=(0, 0, 1,0) 
k = (0, 0, 0,1) 


(pure cyan) 
(pure magenta) 
(pure yellow) 
(pure black) 


in r 4 and describing the ink color as a linear combination of these using coefficients between 0 
and 1, inclusive. Thus, an ink color p is represented as a linear combination of the form 


1> = C 1 c -I- C2in + C3V + C4k — (ci, c 2, C3, cf) 

where 0 <c, < l- The set of all such linear combinations is called CMYK space , although it is not 
a subspace of r 4 . (Why?) For example, Pantone color 876CVC is a mixture of 38% cyan, 59% 
magenta, 73% yellow, and 7% black; Pantone color 216CVC is a mixture of 0% cyan, 83% 
magenta, 34% yellow, and 47% black; and Pantone color 328CVC is a mixture of 100% cyan, 
0% magenta, 47% yellow, and 30% black. We can denote these colors by 
P8 76 = (0.38, 0.59, 0.73, 0.07), p 2 ie = (0, 0.83, 0.34, 0.47), and =(1,0, 0.47, 0.30), respectively. 


Subspaces of Functions Continuous on ( — oo , oo) 


Calculus Required 

Recall from calculus that if/ and g are continuous functions on the interval ( _ oo , oo ) and k is a constant, then f + g and 
)tf are also continuous. Thus the continuous functions on the interval ( _ oo , oo ) form a subspace of p ( _ oo , oo ), since 
they are closed under addition and scalar multiplication. We denote this subspace by C( — oo , oo )• Similarly, if/ and g 
have continuous first derivatives on ( _ oo , oo ), then so do f + g and . Thus the functions with continuous first 
derivatives on ( _ oo , oo ) form a subspace of p ( _ oo , oo )• We denote this subspace by C 1 ( — oo , oo ), where the 
superscript 1 is used to emphasize th e first derivative. However, it is a theorem of calculus that every differentiable function 
is continuous, so C 1 ( — oo , oo ) is actually a subspace of C( — oo , oo )• 

To take this a step further, for each positive integer m, the functions with continuous mth derivatives on ( _ oo , oo ) form a 
subspace of C 1 ( — oo , oo ) as do the functions that have continuous derivatives of all orders. We denote the subspace of 
functions with continuous mth derivatives on(_ oo, oo ) by C m ( — oo, oo), and we denote the subspace of functions that 
have continuous derivatives of all orders on ( _ oo , oo ) by C v ( — oo , oo ). Finally, it is a theorem of calculus that 
polynomials have continuous derivatives of all orders, so p n is a subspace of C '(— do , oo ). The hierarchy of subspaces 
discussed in this example is illustrated in Figure 5.2.4. 






Figure 5.2.4 


Remark In the preceding examplewe focused on the interval ( _ oo , oo )• Had we focused on a closed interval [a, A], 
then the subspaces corresponding to those defined in the example would be denoted by C[a, &], C m [a, b], and C ' [a,b]. 
Similarly, on an open interval (a, b) they would be denoted by C(a, b), C m (a, &), and C ' {a, b) 


Solution Spaces of Homogeneous Systems 

If Ax = h is a system of linear equations, then each vector x that satisfies this equation is called a solution vector of the 
system. The following theorem shows that the solution vectors of a homogeneous linear system form a vector space, which 
we shall call the solution space of the system. 


THEOREM 5.2.2 


If Ax = 0 i s a homogeneous linear system ofm equations in n unknowns, then the set of solution vectors is a subspace of 
R”. 


Let IT be the set of solution vectors. There is at least one vector in W, namely 0 - To show that W is closed under 
addition and scalar multiplication, we must show that if jc and x 1 are any solution vectors and k is any scalar, then x H x ! and 
£x are also solution vectors. But if x and x 1 are solution vectors, then 


from which it follows that 


= 0 and = 0 

j4(x + x 1 ) = j4x | V = 0 + 0 = 0 


and 


^(Ax) =ifcdx = A0 = 0 

which proves that x-i and tx are solution vectors. 


Solution Spaces That Are Subspaces of R 3 

















Consider the linear systems 


'1 -2 3' 

~x " 


'o' 

2-4 6 


= 

0 

3-6 9 

z 


0 


1 -2 

3 

~x " 


'o' 

-3 7 

-8 


= 

0 

^r 

C-l 

1 

_ 1 

-6 

z 


0 


1 

-2 

3 

~x " 


'o' 

-3 

7 

-8 


= 

0 

4 

1 

2 

z 


0 


O 

O 

O 

~x " 


'o' 

000 

y 

= 

0 

000 

z 


0 


Each of these systems has three unknowns, so the solutions form subspaces of j? 3 . Geometrically, this means that each 
solution space must be the origin only, a line through the origin, a plane through the origin, or all of p}. We shall now verify 
that this is so (leaving it to the reader to solve the systems). 


Solution 

(a) The solutions are 


from which it follows that 


x — 2s — 3t, y — s, z — t 


x = 2y — 3z, or x — 2y f 3z = 0 


This is the equation of the plane through the origin with n = (l, - 2 , 3) as a normal vector. 


(b) The solutions are 


x——5t, y — — t, z — t 

which are parametric equations for the line through the origin parallel to the vector 

v=(-5, -1,1). 

(c) The solution is x = 0> y = Ct z = 0» s ° the solution space is the origin only—that is, {0}. 


(d) The solutions are 


x=r, y=s, z = t 

where r, s, and t have arbitrary values, so the solution space is all of p 3 . 



























In Section 1.3 we introduced the concept of a linear combination of column vectors. The following definition extends this 
idea to more general vectors. 


DEFINITION 


A vector w is called a linear combination of the vectors Vlj y 2 i v,. if it can be expressed in the form 

w = jfcivi H ^ 2 v 2 + + kr v r 

where *p * 2 , are scalars. 


If r — 1, then the equation in the preceding definition reduces to w = £ lVl ; that is, w is a linear combination of a 
single vector vq if it is a scalar multiple of vp 


Vectors in R 3 Are Linear Combinations of i, j, and k 


Every vector v = (a, b, c) in P? is expressible as a linear combination of the standard basis vectors 

i= ( 1 , 0 , 0 ), j= ( 0 , 1 , 0 ), k= ( 0 , 0 , 1 ) 

since 

v—{a,b,c)—a{ 1 , 0 , 0 ) 4 - *( 0 , 1 , 0 ) +c( 0 , 0 , 1 ) = ai + *j + ck 

♦ 


Checking a Linear Combination 


Consider the vectors u = (l ; 2, — 1) and y — ( 6 , 4 , 2) in p?. Show that w = (9, 2, 7) is a linear combination of u and v and 
that w ; — (4, —1,8) is not a linear combination of u and v. 


Solution 

In order for w to be a linear combination of u and v, there must be scalars k\ and £ 2 such that w = £ lU -|- kiV, that is, 

(9,2,7)=*i(l,2, — 1) + * 2 ( 6 . 4 P 2) 
or 

(9, 2, 7) = (*i + 6 ^ 2 , 2*i 4 4*2, -*l + 2* 2 ) 

Equating corresponding components gives 

*1 4- 6*2 = 3 
2*i + 4*2 — 2 
— *i 4- 2*2 — 7 

Solving this system using Gaussian elimination yields *j — _ 3 , * 2 = 2, so 

w = — 3u 4 - 2v 











Similarly, for vt J to be a linear combination of u and v, there must be scalars k\ and k 2 such that v/ = .tju -|- k 2 V, that is, 

(4, — 1, 8) =jti(l, 2, -l)+i 2 (6,4,2) 
or 

(4, —1,8) = (*i 4 6k 2 , 2k x 4 4k 2 , -*i 4- 2* 2 ) 

Equating corresponding components gives 

4 6£ 2 = 4 
2^7i 4 4A:2 = — 1 
— Aq 4 2k 2 = 8 

This system of equations is inconsistent (verify), so no such scalars ^ and k 2 exist. Consequently, vi' is not a linear 
combination of u and v. 

♦ 


Spanning 

If vi, \' 2 ,Vj. are vectors in a vector space V, then generally some vectors in V may be linear combinations of y 1; v 2 ,v,. 
and others may not. The following theorem shows that if we construct a set W consisting of all those vectors that are 
expressible as linear combinations of y 1; v 2 , v r then W forms a subspace of V. 


THEOREM 5.2.3 


Ifvi, \' 2 ,..., Vj. are vectors in a vector space V, then 

(a) The set W of all linear combinations of y^ y 2 i y,. is a subspace ofV. 


(b) W is the smallest subspace of V that contains \q, v 2 ,..v,. hi the sense that every other subspace of V that 
contains y 1; y 2j V?I must contain W. 


if (a) To show that IT is a subspace of V, we must prove that it is closed under addition and scalar multiplication. There 
is at least one vector in W —namely 0, since 0 = Ovi 4 0v 2 H_h 0\y If u and v are vectors in IT, then 


u = ciV! 4 C2 v 2 + - T <=r v r 


and 


v = k iv'i 4 k 2 \ 2 H-1- k r v r 

where Cl , c 2 ,c r , k 2 ,k r are scalars. Therefore, 

u 4 V = (c 1 + Aq)vi 4 (c 2 4 k 2 )v 2 4 -4 (c r 4 £ r )v>- 


and, for any scalar k, r 


£u = (jtci)vi 4 (kc 2 )v 2 H-h (fe r )v r 


Thus u + v and kn are linear combinations of Vl , v 2 , v r and consequently lie in 1/1/. Therefore, 1/1/ is 
closed under addition and scalar multiplication. 



Proof (b) Each vector v 3 is a linear combination of Vli v r 2 ,\> since we can write 

Vj = Ovi 4- 0v2 H-h Ivy H-h 0v> 

Therefore, the subspace W contains each of the vectors Vl , v 2 , v r - Let w' be any other subspace 
that contains Vl , v 2 ,v r Since W ! is closed under addition and scalar multiplication, it must contain 
all linear combinations of V2 , v r . Thus, contains each vector of W. 


We make the following definition. 


DEFINITION 


If S= ( Vli v 2i Vj.} is a set of vectors in a vector space V, then the subspace W of V consisting of all linear 
combinations of the vectors in S is called the space spanned by y 1; y 2i v r and we say that the vectors Vli y 2> v> 

span W. To indicate that W is the space spanned by the vectors in the set S = (vi, v 2i v,.}, we write 

W — span {£) or W— span (vi, V 2 ,\>} 


EXAMPLE 10 Spaces Spanned by One or Two Vectors 


If vi and v '2 are noncollinear vectors in g-' with their initial points at the origin, then span { Vl> y 2 ) , which consists of all 
linear combinations ^ lVl | A: 2 v 2 , is the plane determined by vi and V2 (see Figure 5.2.5a). Similarly, if v is a nonzero vector 
in g 2 or g 2 , then span (v), which is the set of all scalar multiples £y, is the line determined by v (see Figure 5.2.5 b). 


(«) Span [Vj.v-| is (he plane through l he 
origin determined by v t and * 


jr 

t,b) Span {vf is the line through I he 
origin determined by i, 



'PJUl [ V |. > , ( 
■+ k ->Vii 



Figure 5.2.5 

♦ 


Spanning Set for p n 















The polynomials l, x,x 2 ,..., x n span the vector space P n defined in Example 5 since each polynomial p in p n can be written 
as 

p = ao T a l x + ■*■-! -a„x” 

which is a linear combination of 1, x, x 2 ,x n . We can denote this by writing 


P n = span {1, x, x 2 ,x ”} 


♦ 


EXAMPLE 12 Three Vectors That Do Not Span R 3 


Determine whether V j = (1, 1, 2)> \'2 = (1, 0, 1), and V 3 = (2, 1, 3) span the vector space j? 3 . 


Solution 

We must determine whether an arbitrary vector — (b\,b 2 , bp) in R 2 can be expressed as a linear combination 

b = & 1 V 1 4- k 2 v 2 T & 3 V 3 

of the vectors vp a nd V 3 . Expressing this equation in terms of components gives 

(b u b 2 , bj) = *i(l, 1, 2) + k 2 (l, 0, 1) + k 3 (2, 1, 3) 
or 

(&i, b 2 , bp) — (£1 + k 2 + 2k 3 , /ti -I- k 3 , 2k\ 4- k 2 + 3 ^ 3 ) 
or 

Aq 4 k 2 T 2^:3 = b\ 

*i + k 2 = b 2 
2k\ q k 2 4 - 3^3 = b 3 


The problem thus reduces to determining whether this system is consistent for all values of b 1 , b 2 , and by By parts (e) and 
(g) of Theorem 4.3.4, this system is consistent for all ij, b 2 -> and b 3 if an d only if the coefficient matrix 


A = 


1 1 2 
1 0 1 
2 1 3 


has a nonzero determinant. However, det(j4) = 0 (verify), so vp v 2 , and V 3 do not span p'-'. 


♦ 


Spanning sets are not unique. For example, any two noncollinear vectors that lie in the plane shown in Figure 5.2.5 will span 
that same plane, and any nonzero vector on the line in that figure will span the same line. We leave the proof of the 
following useful theorem as an exercise. 


THEOREM 5.2.4 


If S— {vp \' 2 ,..., Vj.) and S' = {wp W 2 ,w&} are two sets of vectors in a vector space V, then 

span {vp v 2 , v*} = span{wp w 2 ,w*} 

if and only if each vector in S is a linear combination of those in S ! and each vector in S ! is a 







linear combination of those in S. 


Exercise Set 5.2 



Click here for Just Ask! 


Use Theorem 5.2.1 to determine which of the following are subspaces of p}. 


(a) all vectors of the form (a, 0, 0) 


(b) all vectors of the form ( a , 1, 1) 


(c) all vectors of the form (a, b,c), where b = a-\- c 


(d) all vectors of the form (a, b, c), where b = a -\ c + 1 


(e) all vectors of the form (a, b, 0) 





Use Theorem 5.2.1 to determine which of the following are subspaces of m 2 2 


2 . 

(a) all 2 x 2 matrices with integer entries 

(b) all matrices 

a b 
c d 

where a 4 Hch d = 0 

(c) all 2 X 2 matrices A such that det(A) = 0 

(d) all matrices of the form 

a b 
0 c 

(e) all matrices of the form 

1“ a a ~ 

[ — a —a 

Use Theorem 5.2.1 to determine which of the following are subspaces of p~>. 

3. 

(a) all polynomials ¥a]X f a 3 x 3 for which aQ = 0 

(b) all polynomials aQ \. ai x \ a 2 x 2 I a 2 x 2 for which , ai+a 2 4 fl 3 = 0 

(c) all polynomials f ayX \ a2X 2 \ a2 x 3 f° r which ag, a a 2 , and a 3 are integers 

(d) all polynomials of the form ag 4- ai*, where ag and a\ are real numbers 

Use Theorem 5.2.1 to determine which of the following are subspaces of the space — 00 , 00 ) 

4. 

(a) all/such that / (x) < 0 for all x 


(b) all/such that / (0) = 0 


(c) all/such that / (0) =2 



(d) all constant functions 


(e) all/of the form ^ | £ 2 sm where and £ 2 are real numbers 


Use Theorem 5.2.1 to determine which of the following are subspaces of M nn - 

S. 

(a) all n x n matrices A such that tr(j4) = 0 

(b) all n x n matrices A such that A T = — A 


(c) all n x n matrices A such that the linear system ^ — () has only the trivial solution 

(d) all n xn matrices A such that AB = BA f° r a fixed n-Kn matrix B 


Determine whether the solution space of the system j4x = 0 is a line through the origin, a plane through the origin, or the 
origin only. If it is a plane, find an equation for it; if it is a line, find parametric equations for it. 

( a > r - 1 1 1 " 

A= 3-10 
2 -4-5 



(d) [12-6' 

A= 1 4 4 

3 10 6 


(e) 1 -1 1 ' 

A— 2 — 1 4 
3 1 11 




Which of the following are linear combinations of u = (0, —2,2) and v = (1, 3, — 1)? 


7. 

(a) (2,2,2) 

(b) (3, 1, 5) 

(c) (0,4,5) 

(d) (0,0,0) 

Express the following as linear combinations of u = (2 1 4), v = (1 —1 3), and w = (3 2 5). 

8 . " 

(a) (-9,-7,-15) 

(b) (6, 11,6) 

(c) (0,0,0) 

(d) (7, 8, 9) 

Express the following as linear combinations of pj = 2 + x I Ax' 1 - = 1 — - 4- 3x 2 ^ and p 3 = 3 \- 2x 4- 5x 2 - 

(a) —9 — lx — 15x 2 

(b) 6 4 1 lx I 6x 2 

(c) 0 

(d) 7 + 8x ■ I 9x 2 


10 . 


Which of the following are linear combinations of 


A = 


4 0 

-2 -2 


B = 


1 - 1 

2 3 


C = 


(a) 


6 -8 

-1 -8 



(b) 


0 0 
0 0 


(c) 


6 0 
3 8 


(d) 


-1 5 
7 1 


In each part, determine whether the given vectors span p-'. 

11 . 


(a) vi = (2, 2, 2), v 2 = (0, 0, 3), v 3 = (0, 1, 1) 


(b) vi = (2, - 1, 3), v 2 = (4, 1, 2), v 3 = (8, - 1, 8) 


(c) vi = (3, 1, 4), v 2 = (2, - 3, 5), v 3 = (5, - 2, 9), v 4 (l, 4,-1) 


(d) vi = (1, 2, 6), v 2 = (3, 4, 1), v 3 = (4, 3, 1), v 4 (3, 3, 1) 


Let f — CO s 2 * and g = sin 2 *. Which of the following lie in the space spanned by/and gl 

12 . 


(a) cos2* 

(b) 3 + * 2 

(c) 1 


(d) sin x 


(e) 0 


Determine whether the following polynomials span p^ 

Pl = 1 — * 4 2* 2 , p 2 = 3 4 *, p 3 = 5 — * f 4* 2 , p4 = — 2 — 2* I- 2* 2 

Let V1 = (2, 1, 0, 3), v 2 = (3, - 1, 5, 2), and V3 = ( - 1, 0, 2, 1)- Which of the following vectors are in 
span { vi, v 2 , v 3 } ? 



(a) (2, 3,-7, 3) 


(b) ( 0 , 0 , 0 , 0 ) 

(c) (1, 1, 1, 1) 

(d) (-4, 6,-13, 4) 

Find an equation for the plane spanned by the vectors u = ( — 1, 1, 1) and v = (3, 4, 4). 

15# 

Find parametric equations for the line spanned by the vector u=(3, —2,5)- 

16 . 

Show that the solution vectors of a consistent nonhomogeneous system of m linear equations in n unknowns do not form 

17. a subspace of R n . 

Prove Theorem 5.2.4. 

18 . 

Use Theorem 5.2.4 to show that Vl = (1,6, 4), v 2 = (2, 4, - 1), v 3 = ( - 1, 2, 5), and Wl = (1, - 2, - 5), 

19- W2 _ ( 0 , 3 , 9 ) span the same subspace of j? 3 . 

A line L through the origin in p-' can be represented by parametric equations of the form x = at, y = bt, and z = ct • Use 
20* these equations to show that L is a subspace of p^; that is, show that if Vl = (* i, y i, z\) and y 2 = (^ 2 , y 2 , z 2 ) are points 
on L and k is any real number, then and v\ + v 2 are also points on L. 

21. (For Readers Who Have Studied Calculus) Show that the following sets of functions are subspaces of p( _ oo , oo ) 


(a) all everywhere continuous functions 

(b) all everywhere continuous functions 

(c) all everywhere continuous functions that satisfy f ; 4- 2f = 0 



22. (For Readers Who Have Studied Calculus) Show that the set of continuous functions f = j (*) on \a,b] such that 


is a subspace of C\a, b ]. 


f 

Ja 


f(x)dx 


= 0 


Discussion 

Discovery 


Indicate whether each statement is always true or sometimes false. Justify your answer by 
23. giving a logical argument or a counterexample. 


(a) If = b is any consistent linar system of m equations in n unknowns, then the 
solution set is a subspace of R n . 


(b) If W is a set of one or more vectors from a vector space V, and if £u + v is a vector in 
W for all vectors u and v in W and for all scalars k, then W is a subspace of V. 

(c) If S is a finite set of vectors in a vector space V, then span(S) must be closed under 
addition and scalar multiplication. 

(d) The intersection of two subspaces of a vector space V is also a subspace of V. 

(e) If span(£i) = spanf^)’ th en = %2' 


24. 

(a) Under what conditions will two vectors in g-' span a plane? A line? 

(b) Under what conditions will it be true that span{u} = span (v) ? Explain. 

(c) If = b is a consistent system of m equations in n unknowns, under what conditions 
will it be true that the solution set is a subspace of R nr ! Explain. 

Recall that lines through the origin are subspaces of p?. If is the line y = 7 , is the line 
25* y — _ v, is the union W\ U Wj a subspace of Explain your reasoning. 

26. 

(a) Let M 22 * 3e th e vector space of 2 x 2 matrices. Find four matrices that span M.22- 


MM - 


(b) In words, describe a set of matrices that spans m■ 




We showed in Example 8 that the vectors i, j, k span ^3. However, spanning sets are not 
27* unique. What geometric property must a set of three vectors in j? 3 have if they are to span p} C) . 
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r o In the preceding section we learned that a set of vectors s= {v-i,v 2 , .. ,,v f } 

spans a given vector space V if every vector in V is expressible as a linear 
LIN E A R INDEPENDENCE combination of the vectors in S. In general, there may be more than one way to 

express a vector in V as a linear combination of vectors in a spanning set. In 
this section we shall study conditions under which each vector in V is 
expressible as a linear combination of the spanning vectors in exactly one way. 
Spanning sets with this property play a fundamental role in the study of vector 
spaces. 


DEFINITION 


If s= {vi, v 2 ,Vj.} is a nonempty set of vectors, then the vector equation 

^l v l + ^2 V 2 H- \ k r v r = 0 

has at least one solution, namely 

£i = 0, £ 2 = 0, ..., k r = 0 

If this is the only solution, then S is called a linearly independent set. If there are other solutions, then S is called a linearly 
dependent set. 


A Linearly Dependent Set 


If VI = ( 2 , - 1 , 0 , 3), v 2 = ( 1 , 2 , 5 , - 1 ), and V3 = ( 7 , - 1 , 5, 8), then the set of vectors S= {vi, v 2 , v 3 ) is linearly dependent, 
since 3 Vl 4 . V2 — v 3 = 0 - 


EXAMPLE 2 A Linearly Dependent Set 


The polynomials 


pi = 1 — x, p 2 = 5 -I- 3x — 2x 


form a linearly dependent set in since 3pj _ p 2 + 2p 3 = 0 - 


and 


p 3 = 1 H 3x — x l 


♦ 


Linearly Independent Sets 


Consider the vectors i = ( 1 , 0, 0). j = (0, 1,0)* and k = (0, 0, 1) in J? 3 - In terms of components, the vector equation 

jfcli I- 4- = 0 


becomes 















or, equivalently, 


*l(l. 0, 0) 4- * 2 (0. 1. 0) + £ 3 ( 0 , 0, 1) = (0, 0, 0) 


(*1.*2.*3) = (0.0. 0) 

This implies that = 0 , k 2 = Cu and £3 = 0> so the set S= (i, j, k) is linearly independent. A similar argument can be used to 
show that the vectors 


•1 = ( 1 . 0.0 . 0 ), 

form a linearly independent set in R n . 


*2 = ( 0 . 1.0 . 0 )..., e M = ( 0 , 0,0 . 1 ) 


♦ 


EXAf Determining Linear Independence/Dependence 

Determine whether the vectors 

vi = (1, -2,3), v 2 = (5, 6 , — 1), v 3 =(3,2,1) 
form a linearly dependent set or a linearly independent set. 


Solution 


In terms of components, the vector equation 
becomes 


fcl v l 4- k 2 v 2 + ^ 3 y 3 = 0 


or, equivalently, 


ii (1. - 2. 3) + k 7 (5, 6 . - 1) + £ 3 ( 3 , 2. 1) = (0, 0, 0) 


(£l H 5k 2 4- 3^3, — 2k\ 4- 6^2 4 2k 2 , 3k\ — k 2 4 k 3 ) = (0, 0, 0) 


Equating corresponding components gives 

jfci 4 5k 2 4 3*3 = 0 
— 2k\ 4 6^2 4 2 £ 3 — 0 
3 ^i — £2 + £3 = 0 

Thus \q, \' 2 , and v 3 form a linearly dependent set if this system has a nontrivial solution, or a linearly independent set if it has 
only the trivial solution. Solving this system using Gaussian elimination yields 

£l = - h = ~ £3 = 1 

Thus the system has nontrivial solutions and vj, \' 2 , and v 2 form a linearly dependent set. Alternatively, we could show the 
existence of nontrivial solutions without solving the system by showing that the coefficient matrix has determinant zero and 
consequently is not invertible (verify). 

♦ 


EXAMPLE 5 Linearly Independent Set in p n 

Show that the polynomials 

1, X, x 2 ,..., x” 

form a linearly independent set of vectors in p . 


Solution 






Let p 0 = l ; pj = x , p 2 = x 2 ,..., p H = x n and assume that some linear combination of these polynomials is zero, say 

flnPn + aiPi + awi + - + a»-p»=0 

or, equivalently, 

ag + a\x 4- a 2 X 2 + —I- a n x n = 0 for all x in ( — oo , oo ) 


(1) 


We must show that 


£?0 = a \ —a2 — - — a n — 0 

To see that this is so, recall from algebra that a nonzero polynomial of degree n has at most n distinct roots. But this implies that 

aq = ai — a2 — — = — 0 ; otherwise, it would follow from 1 that ag -|- a\x 4 - 0,2x 2 H_f a n x n is a nonzero polynomial with 

infinitely many roots. 

♦ 


The term linearly dependent suggests that the vectors “depend” on each other in some way. The following theorem shows that 
this is in fact the case. 


THEOREM 5.3.1 


A set S with two or more vectors is 


(a) Linearly dependent if and only if at least one of the vectors in S is expressible as a linear combination of the other 
vectors in S. 


(b) Linearly independent if and only if no vector in S is expressible as a linear combination of the other vectors in S. 


We shall prove part (a) and leave the proof of part ( b ) as an exercise. 


Proof (a) Let S= {vj, \’ 2 ,..., v>} be a set with two or more vectors. If we assume that S is linearly dependent, then there are 
scalars k\, kj,k r , not all zero, such that 


^l v l d ^2 V 2 H-f kr v r = 0 


( 2 ) 


To be specific, suppose that ^ *0- Then 2 can be rewritten as 


VI - 


*2 

*1 


v 2 + ■" + 



r 


which expresses vq as a linear combination of the other vectors in S. Similarly, if kj±0 in 2 for some 
j = 2,3,..., r, then Vj is expressible as a linear combination of the other vectors in S. 

Conversely, let us assume that at least one of the vectors in S is expressible as a linear combination of the other vectors. To be 
specific, suppose that 

vi — C2V2 + C3V3 + - + c r v r 


so 


VI - c 2 v 2 - c 3 V 3- c r v r = 0 


It follows that S is linearly dependent since the equation 

iivi + ^2 v 2 H-1- = 0 


is satisfied by 



= 1, h = — c 2> 


k r — — c r 

which are not all zero. The proof in the case where some vector other than vj is expressible as a linear combination of the other 
vectors in S is similar. 


Example 1 Revisited 


In Example 1 we saw that the vectors 

vi = (2, - 1, 0, 3), v 2 = (1, 2, 5, - 1), and v 3 = (7, - 1, 5, 8 ) 

form a linearly dependent set. It follows from Theorem 5.3.1 that at least one of these vectors is expressible as a linear 
combination of the other two. In this example each vector is expressible as a linear combination of the other two since it follows 
from the equation 3 Vl + \'2 — v 3 — 0 ( see Example 1) that 

vi = — -jV 2 + JV 3 , \’2 = — 3vi 4 v 3 , and v 3 = 3v\ 4 \'2 


Example 3 Revisited 


In Example 3 we saw that the vectors i = (1, 0, 0), j = (0, 1, 0)> and k = (0, 0, 1) form a linearly independent set. Thus it follows 
from Theorem 5.3.1 that none of these vectors is expressible as a linear combination of the other two. To see directly that this is 
so, suppose that k is expressible as 

k = £ii + kf\ 


Then, in terms of components, 

(0, 0, l)=*i(1.0, 0) 4 £ 2 (0, 1,0) 


or (0, 0, 1) = (£i, £ 2 , 0) 


But the last equation is not satisfied by any values of and b-r so ^ cannot be expressed as a linear combination of i and j. 
Similarly, i is not expressible as a linear combination of j and k, and j is not expressible as a linear combination of i and k. 

♦ 


The following theorem gives two simple facts about linear independence that are important to know. 


THEOREM 5.3.2 


(a) A finite set of vectors that contains the zero vector is linearly dependent. 


(b) A set with exactly two vectors is linearly independent if and only if neither vector is a scalar multiple of the other. 


We shall prove part (a) and leave the proof of part ( b ) as an exercise. 


For any vectors Vli v r , the set S= {vq, v 2 ,vv 0 } is linearly dependent since the equation 


Ovi 4 - 0\'2 H-h 0 v r + 1 ( 0 ) = 0 







expresses o as a linear combination of the vectors in S with coefficients that are not all zero. 


Using Theorem 5.3.2 b 


The functions f j = % and f 2 = sin x form a linearly independent set of vectors in F( — oo , oo), since neither function is a 
constant multiple of the other. 

♦ 


Geometric Interpretation of Linear Independence 

Linear independence has some useful geometric interpretations in p} and p}\ 


In p' 1 or p}, a set of two vectors is linearly independent if and only if the vectors do not lie on the same line when they are 
placed with their initial points at the origin (Figure 5.3.1). 



Figure 5.3.1 

In p}, a set of three vectors is linearly independent if and only if the vectors do not lie in the same plane when they are 
placed with their initial points at the origin (Figure 5.3.2). 


Itfj Linearly dependent 

Figure 5.3.2 



(/>) Linearly dependent (r) Linearly independent 




The first result follows from the fact that two vectors are linearly independent if and only if neither vector is a scalar multiple of 
the other. Geometrically, this is equivalent to stating that the vectors do not lie on the same line when they are positioned with 
their initial points at the origin. 

The second result follows from the fact that three vectors are linearly independent if and only if none of the vectors is a linear 
combination of the other two. Geometrically, this is equivalent to stating that none of the vectors lies in the same plane as the 
















other two, or, alternatively, that the three vectors do not lie in a common plane when they are positioned with their initial points 
at the origin (why?). 

The next theorem shows that a linearly independent set in R n can contain at most n vectors. 


THEOREM 5.3.3 


Let S= (v'i, \’ 2 ,..., Vj.) be a set of vectors in Rf 1 . If r > n> then S is linearly dependent. 


Proof Suppose that 


VI = (vn, V12,-..,V1 H ) 
v 2 = ( v 21> v 22> ---> v 2h) 


v r = Om, v r 2 ,..,v m ) 


Consider the equation 


£i v l 4- bos '2 H-1- h r v r = 0 


If, as illustrated in Example 4, we express both sides of this equation in terms of components and 
then equate corresponding components, we obtain the system 

vnjfci 4- V21 k 2 4- - 4 = 0 

Vi2^1 -I V22^2 H-1- v r 2^r = 0 


vih^i 4 v 2 „k 2 4 - 4 v rYl k r = 0 

This is a homogeneous system of n equations in the r unknowns £ 1; k r - Since r>n, it follows from 
Theorem 1.2.1 that the system has nontrivial solutions. Therefore, s= (vi, v 2 ,..., is a linearly 
dependent set. 


Remark The preceding theorem tells us that a set in R 2 with more than two vectors is linearly dependent and a set in pf with 
more than three vectors is linearly dependent. 

Linear Independence of Functions 

Sometimes linear dependence of functions can be deduced from known identities. For example, the functions 

Calculus Required 


■ 2 2 

fl=sm"x, f 2 = cosx, and f? = 5 
form a linearly dependent set in R( _ qg , oo )* since the equation 

5f i I 5f2 — (3 = 5sin 2 x 4 5cos 2 x — 5 = 5(sin 2 x 4 cos 2 x) — 5 = 0 

expresses 0 as a linear combination of f j, f 2 , and f , with coefficients that are not all zero. However, it is only in special 
situations that such identities can be applied. Although there is no general method that can be used to establish linear 
independence or linear dependence of functions in _ 00 , 00 )> we shall now develop a theorem that can sometimes be used to 
show that a given set of functions is linearly independent. 

If f j = f j (x), f 2 = / 2 OO, - f H = /mOO ar| d n — 1 times differentiable functions on the interval ( _ 00 , 00 )> then the 
determinant 



/1 00 fl(x) - /hOO 
/[W / 2 OO - /«0O 




is called the Wronskian of f f 2 ,f n - Aswe shall now show, this determinant is useful for ascertaining whether the 
functions fi.fa . f» form a linearly independent set of vectors in the vector space C l f! 11 ( — do , oo ). 


Suppose, for the moment, that f 1; f 2 ,f H are linearly dependent vectors in C^ 1 ^( — oo , oo ). Then there exist scalars 
k\, k 2 , j k n , not all zero, such that 

*1 / 1 0O +kj..f7(x) +~ + k».f„(x) — 0 

for all x in the interval ( _ oo , oo ) • Combining this equation with the equations obtained by n _ 1 successive differentiations 
yields 

*l/lOO + *2/200 4-b *h/mOO =0 

*1/1 (*) + *2/2^) * h / h 00 =0 


*1/1" ^O+^/S' 5 15 W+- + *„/? 1 \x) = o 

Thus, the linear dependence of f ^ f 2 ,f n implies that the linear system 


/ 100 / 2 OO “/hW 
/ !W 1 2 00 “/wOO 

*1 

*2 


1 

' O O 
_1 


1 - 

rV " 

1_ 


1 

O " 
1_ 


has a nontrivial solution for every x in the interval ( _ 00 , 00 )• This implies in turn that for every x in ( _ 00 , 00 ) the 
coefficient matrix is not invertible, or, equivalently, that its determinant (the Wronskian) is zero for every x in ( _ 00 , do )• 
Thus, if the Wronskian is not identically zero on ( _ 00 , 00 ), then the functions f j, f 2 ,f H must be linearly independent 
vectors in C (M— 0( _ rxj, oc>). This is the content of the following theorem. 



Jozef Maria Hoene-Wronski 


Jozef Maria Hoene-Wronski (1776-1853) was a Polish-French mathematician and philosopher. Wronski received his early 
education in Poznan and Warsaw. He served as an artillery officer in the Prussian army in a national uprising in 1794, was 










taken prisoner by the Russian army, and on his release studied philosophy at various German universities. He became a French 
citizen in 1800 and eventually settled in Paris, where he did research in analysis leading to some controversial mathematical 
papers and relatedly to a famous court trial over financial matters. Several years thereafter, his proposed research on the 
determination of longitude at sea was rebuffed by the British Board of Longitude, and Wronski turned to studies in Messianic 
philosophy. In the 1830s he investigated the feasibility of caterpillar vehicles to compete with trains, with no luck, and spent 
his last years in poverty. Much of his mathematical work was fraught with errors and imprecision, but it often contained 
valuable isolated results and ideas. Some writers attribute this lifelong pattern of argumentation to psychopathic tendencies 
and to an exaggeration of the importance of his own work. 


THEOREM 5.3.4 


If the functions f 1; f 2> ..., f „ have n — \ continuous derivatives on the interval ( _ oo , do )> and if the Wronskian of these 
functions is not identically zero on oo , do ). then these functions form a linearly independent set of vectors in 
C'( M-1 )( _ oo , oo ). 


Linearly Independent Set in C 1 (— oo , co) 


Show that the functions f j = x and f 2 = sin x form a linearly independent set of vectors in C 1 ( — oo , do ). 


Solution 


In Example 8 we showed that these vectors form a linearly independent set by noting that neither vector is a scalar multiple of the 
other. However, for illustrative purposes, we shall obtain this same result using Theorem 5.3.4. The Wronskian is 


W(x) = 


sin x 
cos x 


= xcos x — sin x 


This function does not have value zero for all x in the interval ( — oo , do ), as can be seen by evaluating it at % = ^ j 2, so f j and 
f 2 form a linearly independent set. 

♦ 


Linearly Independent Set in C 2 ( — oo , oo ) 


Show that fj — j, f 2 = e x , and f 3 = e " 2x form a linearly independent set of vectors in C 2 ( — oo , oo ). 


Solution 

The Wronskian is 


1 e x e 2x 


W(x)= u a Xr >s 2x =2e 

This function does not have value zero for all x (in fact, for any x) in the interval ( _ oo , oo ), so f j, f 2 , and f 3 form a linearly 


3j 











independent set. 


♦ 


The converse of Theorem 5.3.4 is false. If the Wronskian of f lr f 2i f H is identically zero on ( _ oo, oo ), then no 
conclusion can be reached about the linear independence of {f j, f 2 ,.... f ; this set of vectors may be linearly independent or 
linearly dependent. 


Exercise Set 5.3 


& 


Click here for Just Ask! 


1 . 


Explain why the following are linearly dependent sets of vectors. (Solve this problem by inspection.) 


(a) m = ( - 1, 2, 4) and „ 2 = (5, - 10, - 20) in R- 


(b) ui = (3, -l).u 2 =(4 > 5),ii 3 = (-4 > 7)in5 2 


(c) in = 3 — 2x + x 2 and p 2 = 6 — 4x 4 2x 2 ™ P 2 


^A = 


-3 4 
2 0 


and B = 


3 

-2 


in 


Mjl 


Which of the following sets of vectors in r 3 are linearly dependent? 

(a) (4, - 1, 2), (-4, 10, 2) 

(b) (-3,0,4), (5, -1,2), (1, 1,3) 

(c) (8, - 1, 3), (4, 0, 1) 

(d) ( - 2, 0, 1), (3, 2, 5), (6, - 1, 1), (7, 0, - 2) 

Which of the following sets of vectors in R 4 are linearly dependent? 
(a) (3, 8, 7, - 3), (1, 5, 3, - 1), (2, - 1, 2, 6), (1, 4, 0, 3) 


(b) (0,0, 2, 2), (3, 3, 0,0), (1,1,0, - 1) 







(c) (0,3, -3, - 6 ), (-2,0,0, - 6 ), (0, -4, -2, -2), (0, -8,4, -4) 


(d) (3, 0, - 3, 6 ), (0, 2, 3, 1), (0, - 2, - 2, 0), ( - 2, 1, 2, 1) 

Which of the following sets of vectors in p are linearly dependent? 

4. 

(a) 2-*4 4* 2 , 3 f 6x 4 lx 2 , 2 4 10*-4 * 2 

(b) 3fr4 x 2 , 2-x I 5x 2 ,4-3x 2 

(c) 6 -x 2 , 1 4 x 4 4x 2 

(d) 1 4 3x 4 3x 2 , x I 4x 2 , 5 4 6x 4 3x 2 , 7 f 2x-x 2 

Assume that vj, v 2 , and V 3 are vectors in p} that have their initial points at the origin. In each part, determine whether the 
three vectors lie in a plane. 

(a) vi = (2, -2, 0 ),v 2 = ( 6 , 1, 4), v 3 = (2, 0, -4) 

(b) vi = ( - 6 , 7, 2), Y 2 = (3, 2, 4), v 3 = (4, - 1, 2) 

Assume that vj, v 2 , and v 3 are vectors in p} that have their initial points at the origin. In each part, determine whether the 
6 * three vectors lie on the same line. 

(a) vi = ( — 1, 2, 3), v 2 = (2, -4, - 6 ), v 3 = ( - 3, 6 , 0) 

(b) vi = (2, - 1, 4), v 2 = (4, 2, 3), v 3 = (2, 7, - 6 ) 

(c) vi = (4, 6 , 8), v 2 = (2, 3, 4), v 3 = ( — 2, -3,-4) 

7. 

(a) Show that the vectors vj = (0, 3, 1, — 1)> v 2 = ( 6 , 0, 5, 1)* and v 3 = (4, — 7, 1, 3) form a linearly dependent set in p4 


(b) Express each vector as a linear combination of the other two. 




8 . 


(a) Show that the vectors vj = (1, 2, 3, 4), V 2 = (0, 1, 0, — 1), and v 3 = (1, 3, 3, 3), form a linearly dependent set in j? 4 . 


(b) Express each vector as a linear combination of the other two. 


9. 


For which real values of A do the following vectors form a linearly dependent set in f,'-''! 


vi 


M-4 


V2= -1). v 3 =( —i -l.A 


Show that if (vj, v 2 , v 3 j is a linearly independent set of vectors, then so are {vi,v 2 }, {\ r i, v 3 j , (v 2 , v 3 j, {v r i), {v 2 } , 
10 * and {v 3 } • 


11 . 


Show that if S= {vj, V 2 ,v r } is a linearly independent set of vectors, then so is every nonempty subset of S. 


12 . 


13. 


14. 


15. 


Show that if (yj, V 2 , v 3 } is a linearly dependent set of vectors in a vector space V, and is any vector in V, then 
{vj, V 2 , v 3 , V 4 } is also linearly dependent. 

Show that if {vi, V 2 ,v?-} is a linearly dependent set of vectors in a vector space V, and if v„ are any vectors in 

V, then {v 1; v 2 ,..., y„ v, +1 ,..„ y„} is also linearly dependent. 

Show that every set with more than three vectors from p 2 is linearly dependent. 


Show that if {vi,V 2 ) is linearly independent and v 3 does not lie in span {y liV 2 },then {vi,V 2 , v 3 } is linearly independent. 


Prove: For any vectors «, v, and w, the vectors u _ v , v _ w , and w — u form a linearly dependent set. 

16. 

Prove: The space spanned by two vectors in j ? 3 is a line through the origin, a plane through the origin, or the origin itself. 

17. 

Under what conditions is a set with one vector linearly independent? 

18. 

Are the vectors vj, v 2 , and v 3 in part (a) of the accompanying figure linearly independent? What about those in part (b)l 
Explain. 



19 . 














Figure Ex-19 


Use appropriate identities, where required, to determine which of the following sets of vectors in M _ oo ) are linearly 
2®* dependent. 

(a) 6, 3sin 2 *, 2cos 2 * 

(b) x, cos x 

(c) 1, sin x, sin2x 

(d) cos2x, sin 2 *, cos 2 * 

(e) (3 -x) 2 ,x 2 -6x,5 

(f) 0, COS 3 7T*, sin^TTX 

21. (For Readers Who Have Studied Calculus) Use the Wronskian to show that the following sets of vectors are linearly 
independent. 

(a) 1 ,*, e x 

(b) sin x, cos x,x sin * 

(c) e x .xe x .x 2 e* 

( d ) l ,*,* 2 

Use part (a) of Theorem 5.3.1 to prove part ( b ). 

22 . 

Prove part (b) of Theorem 5.3.2. 

23. 


Discussion 

Discovery 


24 . 


Indicate whether each statement is always true or sometimes false. Justify your answer by giving 
logical argument or a counterexample. 




(a) The set of 2 x 2 matrices that contain exactly two l's and two 0's is a linearly independent 
set in Mji- 

(b) If {vi, \ r 2 ) is a linearly dependent set, then each vector is a scalar multiple of the other. 

(c) If (vi,V 2 , V 3 } is a linearly independent set, then so is the set .tv?, jfcv' 3 } for every 
nonzero scalar k. 

(d) The converse of Theorem 5.3.2 a is also true. 


Show that if (vj, V 2 , V 3 } is a linearly dependent set with nonzero vectors, then each vector in the 
25* set is expressible as a linear combination of the other two. 

Theorem 5.3.3 implies that four nonzero vectors in j ? 3 must be linearly dependent. Give an 
26* informal geometric argument to explain this result. 


27. 

(a) In Example 3 we showed that the mutually orthogonal vectors i, |, and fc form a linearly 
independent set of vectors in j? 3 . Do you think that every set of three nonzero mutually 
orthogonal vectors in p} is linearly independent? Justify your conclusion with a geometric 
argument. 

(b) Justify your conclusion with an algebraic argument. 


Hint Use dot products. 


Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 



5.4 

BASIS AND DIMENSION 


We usually think of a line as being one-dimensional, a plane as 
two-dimensional, and the space around us as three-dimensional. It is the 
primary purpose of this section to make this intuitive notion of “dimension” more 
precise. 


Nonrectangular Coordinate Systems 


In plane analytic geometry we learned to associate a point P in the plane with a pair of coordinates (a, b) by projecting P onto a 
pair of perpendicular coordinate axes (Figure 5.4.1a). By this process, each point in the plane is assigned a unique set of 
coordinates, and conversely, each pair of coordinates is associated with a unique point in the plane. We describe this by saying that 
the coordinate system establishes a one-to-one correspondence between points in the plane and ordered pairs of real numbers. 
Although perpendicular coordinate axes are the most common, any two nonparallel lines can be used to define a coordinate system 
in the plane. For example, in Figure 5.4. lb, we have attached a pair of coordinates ( a , b) to the point P by projecting P parallel to 
the nonperpendicular coordinate axes. Similarly, in 3-space any three noncoplanar coordinate axes can be used to define a 
coordinate system (Figure 5.4.1c). 




la} Coordinates of P in a rectangular |7q Coordinates of/ 1 in a 

coordinate system in 2-space nonrectangular coordinate 

system in 2-spaee 



(t) Coordinates off in a 
nonrecta ngular coord i nate 
system in 3-space 


Figure 5.4.1 


Our first objective in this section is to extend the concept of a coordinate system to general vector spaces. As a start, it will be 
helpful to reformulate the notion of a coordinate system in 2-space or 3-space using vectors rather than coordinate axes to specify 
the coordinate system. This can be done by replacing each coordinate axis with a vector of length 1 that points in the positive 
direction of the axis. In Figure 5.4.2a, for example, \q and \ r 2 are such vectors. As illustrated in that figure, if P is any point in the 
plane, the vector Qp can be written as a linear combination of \q and V 2 by projecting P parallel to vj and \ r 2 to make Qp the 
diagonal of a parallelogram determined by vectors a\\ and fop 

OP = a\q F bv 2 

It is evident that the numbers a and b in this vector formula are precisely the coordinates of P in the coordinate system of Figure 
5.4. lb. Similarly, the coordinates ( a , b, c) of the point P in Figure 5.4.1c can be obtained by expressing qp as a linear 
combination of the vectors shown in Figure 5A.2b. 














(h) 


Figure 5.4.2 

Informally stated, vectors that specify a coordinate system are called “basis vectors” for that system. Although we used basis 
vectors of length 1 in the preceding discussion, we shall see in a moment that this is not essential—nonzero vectors of any length 
will suffice. 

The scales of measurement along the coordinate axes are essential ingredients of any coordinate system. Usually, one tries to use 
the same scale on each axis and to have the integer points on the axes spaced 1 unit of distance apart. However, this is not always 
practical or appropriate: Unequal scales or scales in which the integral points are more or less than 1 unit apart may be required to 
fit a particular graph on a printed page or to represent physical quantities with diverse units in the same coordinate system (time in 
seconds on one axis and temperature in hundreds of degrees on another, for example). When a coordinate system is specified by a 
set of basis vectors, then the lengths of those vectors correspond to the distances between successive integer points on the 
coordinate axes (Figure 5.4.3). Thus it is the directions of the basis vectors that define the positive directions of the coordinate axes 
and the lengths of the basis vectors that establish the scales of measurement. 
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(w) Equal scales. Perpendicular axes. 
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[h\ Unequal scales. Perpendicular axes. 



Figure 5.4.3 

The following key definition will make the preceding ideas more precise and enable us to extend the concept of a coordinate 
system to general vector spaces. 


DEFINITION 


If V is any vector space and S= {vj, V2, ..., v„} is a set of vectors in V, then S is called a basis for V if the following two 
conditions hold: 

(a) S is linearly independent. 

(b) S spans V. 


A basis is the vector space generalization of a coordinate system in 2-space and 3-space. The following theorem will help us to see 
why this is so. 


THEOREM 5.4.1 





















Uniqueness of Basis Representation 

IfS= {vi, V 2 , v H } is a basis for a vector space V, then every vector v in V can be expressed in the form 
v = civi + C2 v 2 H-I -c n v n in exactly one way. 


Proof Since S spans V, it follows from the definition of a spanning set that every vector in V is expressible as a linear 
combination of the vectors in S. To see that there is only one way to express a vector as a linear combination of the vectors in S, 
suppose that some vector v can be written as 


V = Cl vi + C 2 v 2 + - 4- c h t« 

and also as 

v = i 1 vi 4 k 2 \’2 H-1 k n Y n 

Subtracting the second equation from the first gives 

0 = (ci -jfci)vi + (c 2 -k 2 )Y 2 + . 4 (c„ 

Since the right side of this equation is a linear combination of vectors in S, the linear independence of S 
implies that 

Cl — jti = 0, c 2 — k 2 = 0, c yi — k yi = 0 

that is, 

C 1 ^1 r k 2 , - - -1 Cyj kyj 

Thus, the two expressions for v are the same. 

■ 

Coordinates Relative to a Basis 


If s = {vi, V 2 , v H } is a basis for a vector space V, and 

v = c ivi + C 2 V 2 -I- - + c„v„ 

is the expression for a vector v in terms of the basis S, then the scalars c\,c 2 , ...,c n are called the coordinates of v relative to the 
basis S. The vector (cj, c 2 , ■■■, c n ) in R n constructed from these coordinates is called the coordinate vector of v relative to S; it is 
denoted by 

( y )s= C c l> c 2> —> c n) 


Remark It should be noted that coordinate vectors depend not only on the basis S but also on the order in which the basis vectors 
are written; a change in the order of the basis vectors results in a corresponding change of order for the entries in the coordinate 
vectors. 


Standard Basis for ft 3 


In Example 3 of the preceding section, we showed that if 

i= (1.0,0), j= (0,1,0), and k= (0,0,1) 

then S— (i, j, k) is a linearly independent set in j? 3 . This set also spans j? 3 since any vector v = (a, b, c) in R n can be written as 

v= (a, b, c) =a(l, 0, 0) +£(0, 1, 0) -\ c(0, 0, 1) =ai + ij + ck 


( 1 ) 





Thus S is a basis for it is called the standard basis for Looking at the coefficients of i,j, and k in 1, it follows that the 
coordinates of v relative to the standard basis are a, b, and c, so 

(vis = (a, b, c) 


Comparing this result to 1, we see that 


V = (v)s 


This equation states that the components of a vector v relative to a rectangular xyz-coordinate system and the coordinates of v 
relative to the standard basis are the same; thus, the coordinate system and the basis produce precisely the same one-to-one 
correspondence between points in 3-space and ordered triples of real numbers (Figure 5.4.4). 



♦ 


The results in the preceding example are a special case of those in the next example. 


Standard Basis for R n 


In Example 3 of the preceding section, we showed that if 

*1 = ( 1 . 0.0 . 0 ), * 2 = ( 0 , 1,0 . 0 ). e „ = ( 0 . 0 , 0 . 1 ) 

then 

S= {<?i, <? 2 ,..., <?„} 

is a linearly independent set in R n . Moreover, this set also spans R n since any vector v = (vj, V 2 ,..., v M ) in R n can be written as 


v = viei, V 2?2 4- 1 -v„e„ 

Thus S is a basis for R n ; it is called the standard basis for R n . It follows from 2 that the coordinates of v = (v i, V 2 ,..., v„) relative 
to the standard basis are vj, v 2 ,v H > so 

(v) s = (vi, v 2 ,.-, v„) 

As in Example 1, we have v = (v)s, so a vector v and its coordinate vector relative to the standard basis for R n are the same. 

♦ 


Remark We will see in a subsequent example that a vector and its coordinate vector need not be the same; the equality that we 
observed in the two preceding examples is a special situation that occurs only with the standard basis for R n . 


In r 2 and r}, the standard basis vectors are commonly denoted by i,j, and k, rather than by ej, anc * f 3 . We shall use 
both notations, depending on the particular situation. 










Demonstrating That a Set of Vectors Is a Basis 


Let vi = (1, 2, 1), v 2 = (2, 9, 0), and V3 = (3, 3, 4)- Show that the set S= { Vl> v 2 , V 3 } is a basis for r 3 . 


Solution 

To show that the set S spans R 3 , we must show that an arbitrary vector b = {b\, b 2 , b 3 ) can be expressed as a linear combination 

b = civi fqvj 4 C3V3 

of the vectors in S. Expressing this equation in terms of components gives 

(b u b 2 , b 3 ) = ci(l, 2, 1) 4 c 2 (2, 9, 0) 4 c 3 (3. 3, 4) 
or 

(& 1 , b 2 , b 3 ) = (ci f 2c 2 -I- 3 c 3 , 2c\ 4 9c 2 4 3c 2 , c\ 4 4c 3 ) 
or, on equating corresponding components, 

c 1 4- 2 c 2 + 3 c 3 = b\ 

2c 1 + 9c 2 + 3c 3 = b 2 ( 3 ) 

ci 4-4c 3 = b 3 

Thus, to show that S spans R 3 , we must demonstrate that system 3 has a solution for all choices of l : , = (h 1 , b 2 , b 3 )- 


To prove that S is linearly independent, we must show that the only solution of 

civi 4- c 2 v 2 4- C 3 V 3 = 0 


(4) 


is cj = c 2 = c 3 = 0- As above, if 4 is expressed in terms of components, the verification of independence reduces to showing that 
the homogeneous system 

ci 4 2c 2 4 3c3 = 0 

2c 1 4 9c 2 4 3 c 3 = 0 ( 5 ) 

ci 4 4c 3 = 0 


has only the trivial solution. Observe that systems 3 and 5 have the same coefficient matrix. Thus, by parts ( b ), (e), and (g) of 
Theorem 4.3.4, we can simultaneously prove that S is linearly independent and spans R 3 by demonstrating that in systems 3 and 5, 
the matrix of coefficients has a nonzero determinant. From 



'1 2 3' 


1 2 3 

A = 

2 9 3 

1 0 4 

we find det(j4) = 

2 9 3 

1 0 4 


and so S is a basis for R 3 . 


♦ 


XA Representing a Vector Using Two Bases 


Let S'= (vi, v 2 , V 3 } be the basis for r 3 in the preceding example. 

(a) Find the coordinate vector ofy=(5, —1,9) with respect to S. 


(b) Find the vector v in r 3 whose coordinate vector with respect to the basis S is (y) = ( — 1, 3, 2)- 










Solution (a) 

We must find scalars ci, c 2 , C 3 such that 


or, in terms of components, 


v — civ 1 -I- c 2 v2 4 - c3V3 


(5, - 1, 9) =ci(l, 2, 1) + c 2 (2, 9, 0) + c 3 (3, 3,4 ) 


Equating corresponding components gives 

ci + 2c 2 + 3c3 =5 

2c 1 4- 9c 2 4- 3c3 = — 1 
ci 4-4c 3 =9 

Solving this system, we obtain C i = L c 2 = — L c 3 = 2 (verify). Therefore, 

(v)c=(l, -1,2) 


Solution (b) 

Using the definition of the coordinate vector ( v ) , we obtain 

v = (— l)vi 4- 3v 2 4- 2v r 3 

= ( - 1)(1, 2, 1) + 3(2, 9, 0) + 2(3, 3, 4) = (11, 31, 7) 


EXAMPLE 5 Standard Basis for p n 


(a) Show that S = { 1, x, x 2 ,..., x n } is a basis for the vector space p n of polynomials of the form | a\x 4 - 1 a n x 

(b) Find the coordinate vector of the polynomial p = a$+a\x 4- fl 2 ?; 2 relative to the basis S= ( 1 , 7 ?) for p 2 - 

Solution (a) 

We showed that S spans p n in Example 11 of Section 5.2, and we showed that S is a linearly independent set in Example 5 of 
Section 5.3. Thus S is a basis for p n , it is called the standard basis for p . 

Solution (b) 

The coordinates of p = q. a ^ x 4 . are the scalar coefficients of the basis vectors 1, x, and so (p) = (ciq, a\, af)- 

♦ 


Standard Basis for M mn 


Let 






1 1 

0 0 

1 —« 0 

1_1 

II 

r-i 

1 1 

1 — • O 

O O 

1 _ 1 

LO 

II 

1 1 

O O 

O i— 1 

1 _ 1 

II 

1-1 

O T— 

O O 

1 _ 1 


Mi = 

The set S'= {M\, M2, M3, M4} is a basis for the vector space Mji of 2 x 2 matrices. To see that S spans M22’ note that an 
arbitrary vector (matrix) 

a b 


c d 


can be written as 


a b 
c d 


= a 


1 0 
0 0 


+ b 


0 1 
0 0 


4 c 


0 0 
1 0 


+ d 


0 0 
0 1 


= aM\ 4 bMj 4 cMi -\- dM^ 

To see that S is linearly independent, assume that 

aMi 4 - bMj 4 CM3 4 dM$ = 0 

That is. 


1 0 
0 0 


4 b 


0 1 
0 0 


4 c 


0 0 
1 0 


4 d 


0 0 
0 1 


0 0 
0 0 


It follows that 


a b 
c d 


0 0 
0 0 


Thus a = b = c = d = 0 > s ° S is linearly independent. The basis S in this example is called the standard basis for M22- More 
generally, the standard basis for M mn consists of the mn different matrices with a single 1 and zeros for the remaining entries. 


Basis for the Subspace span(S) 


If S = {vi, V2,..., \>} is a linearly independent set in a vector space V, then S is a basis for the subspace span( 5 ) since the set S 
spans span(S) by definition of span(S). 


DEFINITION 


A nonzero vector space V is called^ finite-dimensional if it contains a finite set of vectors ( Vli V2,..., v H } that forms a basis. If 
no such set exists, V is called infinite-dimensional. In addition, we shall regard the zero vector space to be finite dimensional. 


Some Finite- and Infinite-Dimensional Spaces 


By Examples Example 2 , Example 5 , and Example 6, the vector spaces R n , p n , and M mn are finite-dimensional. The vector spaces 
F(— 00 , 00 ), C( — 00 , 00 )? C" M ( — 00 , 00 ), and C v ( — 00 , 00 ) are infinite-dimensional (Exercise 24 ). 

♦ 

The next theorem will provide the key to the concept of dimension. 


THEOREM 5.4.2 











































Let V be a finite-dimensional vector space, and let (y 1; y 3 ,y M } be any basis. 
(a) If a set has more than n vectors, then it is linearly dependent. 


(b) If a set has fewer than n vectors, then it does not span V. 


Proof (a) Let S' = {wj, W 2 ,w m ) be any set of m vectors in V, where m > n . We want to show that S ! is linearly dependent. 
Since s = {vj, w 2 , v H ) is a basis, each W; can be expressed as a linear combination of the vectors in S, say 


W 1 =fliivi 4 a 2 \V2 +■•■+ 
w 2 = a 12 v l 4 a%F2 +“4- a„ 2 \ n 

= fliBiVi 4 a 2m v 2 + - 4- a nm v n 

To show that S 1 is linearly dependent, we must find scalars ki,k 2 ,...,k mt not all zero, such that 

£iwi 4- k 2 w 2 4- - -I k m w m = 0 

Using the equations in 6, we can rewrite 7 as 

Ctlfln +k 2 a n +“4-i ffl oi m )vi 

+ (ilfl21 4 k 2<*22 + + k m a 2m) v 2 


( 6 ) 


(7) 


4" (^I^hI I k 2 a n2^ b —0 


Thus, from the linear independence of S, the problem of proving that S ! is a linearly dependent set 
reduces to showing there are scalars k\,k 2 ,...,k m , not all zero, that satisfy 

<311*1 + <312*2 + —b a lm k m = 0 
<321*1 + &22 k 2 4 h a 2m k m = 0 


<3^1*1 I I I a ysm k m — 0 


But 8 has more unknowns than equations, so the proof is complete since Theorem 1.2.1 guarantees the 
existence of nontrivial solutions. 


Let S' = (wi, W 2 , w ffl } be any set of m vectors in V, where m < n . We want to show that S'' does not span V. The 
proof will be by contradiction: We will show that assuming S r spans V leads to a contradiction of the linear independence of 

{ V1, V 2 , ..V jj } • 


If S ! spans V, then every vector in V is a linear combination of the vectors in S ! ■ In particular, each basis vector y,- is a linear 
combination of the vectors in S', say 

vi =flnwi 4- a 2 iw 2 + ~+a m ivt m 

v 2 = a 12 w l + 322*2 + - + a m2 v 'm 

v H = a i„wi + a 2yi Yi 2 + - + a mn vr m 

To obtain our contradiction, we will show that there are scalars k 2 ,k n , not all zero, such that 


(9) 


4- k 2 v 2 + 4- k n Y„ = 0 


( 10 ) 



But observe that 9 and 10 have the same form as 6 and 7 except that m and n are interchanged and the w's and v's are interchanged. 
Thus the computations that led to 8 now yield 

a\\k\ H ^12^1 H-1 ^1h^« =0 

321*1 H <*22*2 H-1 £? 2h*« — 0 

“i fl m2*2 "I I = 0 

This linear system has more unknowns than equations and hence has nontrivial solutions by Theorem 1.2.1. 

■ 

It follows from the preceding theorem that if s = {\q, v r 2 ,..., v H ) is any basis for a vector space V, then all sets in V that 
simultaneously span V and are linearly independent must have precisely n vectors. Thus, all bases for V must have the same 
number of vectors as the arbitrary basis S. This yields the following result, which is one of the most important in linear algebra. 


THEOREM 5.4.3 


All bases for a finite-dimensional vector space have the same number of vectors. 


To see how this theorem is related to the concept of “dimension,” recall that the standard basis for R n has n vectors (Example 2). 
Thus Theorem 5.4.3 implies that all bases for Rfi have n vectors. In particular, every basis for p} has three vectors, every basis for 
has two vectors, and every basis for 3? 1 ( = 3?) has one vector. Intuitively, p 3 is three-dimensional, (a plane) is 

two-dimensional, and R (a line) is one-dimensional. Thus, for familiar vector spaces, the number of vectors in a basis is the same 
as the dimension. This suggests the following definition. 


DEFINITION 


The dimension of a finite-dimensional vector space V, denoted by dim( V), is defined to be the number of vectors in a basis for 
V. In addition, we define the zero vector space to have dimension zero. 


Remark 

From here on we shall follow a common convention of regarding the empty set to be a basis for the zero vector space. This is 
consistent with the preceding definition, since the empty set has no vectors and the zero vector space has dimension zero. 


Dimensions of Some Vector Spaces 


dim(i? M ) = n 
dim(P H ) = n -V 1 
= mn 


[Tht standard basis has n vectors (Example 2).] 
[The standard basis has n + 1 vectors (Example 5).] 
[The standard basis has mn vectors (Example 6).] 


Dimension of a Solution Space 











Determine a basis for and the dimension of the solution space of the homogeneous system 

2x\ 4 2x2 ~ x 3 4- x$ = 0 
— x\ —X 2 + 2 x 2 ~ 3*4 f xj = 0 
x\+X2~2x2 — X;5 = 0 

X2 + *4 + x$ = 0 


Solution 

In Example 7 of Section 1.2 it was shown that the general solution of the given system is 

xj = — s — t, X 2 = s, X 2 = —t, *4 = 0, x$ = t 

Therefore, the solution vectors can be written as 


"*l" 


— s — t 


" — s" 


-£ 


’ - f 


’ - l" 

x 2 


s 


s 


0 


i 


0 

x 3 

= 

— £ 

= 

0 

4 

-£ 

= s 

0 

4 1 

-1 

x 4 


0 


0 


0 


0 


0 

*5 


£ 


0 


£ 


0 


1 


which shows that the vectors 


’ -1" 


’ - f 

1 


0 

0 

and vi = 

-1 

0 


0 

0 


1 


span the solution space. Since they are also linearly independent (verify), {vj, V 2 } is a basis, and the solution space is 
two-dimensional. 


* 


Some Fundamental Theorems 

We shall devote the remainder of this section to a series of theorems that reveal the subtle interrelationships among the concepts of 
spanning, linear independence, basis, and dimension. These theorems are not idle exercises in mathematical theory—they are 
essential to the understanding of vector spaces, and many practical applications of linear algebra build on them. 

The following theorem, which we call the Plus/Minus Theorem (our own name), establishes two basic principles on which most of 
the theorems to follow will rely. 


THEOREM 5.4.4 


Plus/Minus Theorem 

Let S be a nonempty set of vectors in a vector space V. 

(a) If S is a linearly independent set, and ifv is a vector in V that is outside of span(S), then the set S LJ (v) that results by 
inserting v into S is still linearly independent. 

(b) Ifv is a vector in S that is expressible as a linear combination of other vectors in S, and if, S'— (v) denotes the set 
obtained by removing v from S, then S and S— (v) span the same space; that is, 

span(£^) — spanks' — (v) ) 




















We shall defer the proof to the end of the section, so that we may move more immediately to the consequences of the theorem. 
However, the theorem can be visualized in R 3 as follows: 


(a) A set S of two linearly independent vectors in pf spans a plane through the origin. If we enlarge S by inserting any vector v 
outside of this plane (Figure 5.4.5a), then the resulting set of three vectors is still linearly independent since none of the 
three vectors lies in the same plane as the other two. 


J-J 



(a) None of the three (/>) Any of the vectors can (cl liither of the coltinear vectors 

vectors lies in the be removed, and the can be removed, and the 

same plane as the remaining two will still remaining two will si ill 

oilier two* span the plane. span the plane. 

Figure 5.4.5 

(b) If S is a set of three noncollinear vectors in pf that lie in a common plane through the origin (Figure 5.4.5/?, c), then the 
three vectors span the plane. However, if we remove from S any vector v that is a linear combination of the other two, then 
the remaining set of two vectors still spans the plane. 


In general, to show that a set of vectors ( Vli v ?,v M ) is a basis for a vector space V, we must showthat the vectors are linearly 
independent and span V. However, if we happen to know that V has dimension n (so that (yj, \' 2 ,..., v M ) contains the right 
number of vectors for a basis), then it suffices to check either linear independence or spanning—the remaining condition will hold 
automatically. This is the content of the following theorem. 


THEOREM 5.4.5 


IfV is an n-dimensional vector space, and if S is a set in V with exactly n vectors, then S is a basis for V if either S spans V or S 
is linearly independent. 


Proof Assume that S has exactly n vectors and spans V. To prove that S is a basis, we must show that S is a linearly independent 
set. But if this is not so, then some vector v in S is a linear combination of the remaining vectors. If we remove this vector from S, 
then it follows from the Plus/Minus Theorem (Theorem 5.4.4/?) that the remaining set of n — 1 vectors still spans V. But this is 
impossible, since it follows from Theorem 5.4.2 b that no set with fewer than n vectors can span an n-dimensional vector space. 
Thus S is linearly independent. 

Assume that S has exactly n vectors and is a linearly independent set. To prove that S is a basis, we must show that S spans V. But 
if this is not so, then there is some vector v in V that is not in span(S). If we insert this vector into S, then it follows from the 
Plus/Minus Theorem (Theorem 5.4.4a) that this set of « + 1 vectors is still linearly independent. But this is impossible, since it 
follows from Theorem 5.4.2a that no set with more than n vectors in an n-dimensional vector space can be linearly independent. 
Thus S spans V. 




Checking for a Basis 


(a) Show that V j = ( — 3, 7) and y 2 = (5, 5) form a basis for p} by inspection. 

(b) Show that Vl = (2, 0, — 1). V 2 = (4, 0, 7), and V 3 = ( — 1, 1, 4) form a basis for ^3 by inspection. 


Solution (a) 

Since neither vector is a scalar multiple of the other, the two vectors form a linearly independent set in the two-dimensional space 
pfi, and hence they form a basis by Theorem 5.4.5. 

Solution (b) 

The vectors vj and \ r 2 form a linearly independent set in the * z -plane (why?). The vector V 3 is outside of the ^ z -plane, so the set 
(vi, \” 2 , V 3 } is also linearly independent. Since pf is three-dimensional, Theorem 5.4.5 implies that (y 1; yj, y 3 ) is a basis for pf. 

♦ 


The following theorem shows that for a finite-dimensional vector space V , every set that spans V contains a basis for V within it, 
and every linearly independent set in V is part of some basis for V. 


THEOREM 5.4.6 


Let S be a finite set of vectors in a finite-dimensional vector space V. 

(a) If S spans V but is not a basis for V, then S can be reduced to a basis for V by removing appropriate vectors from S. 


(b) If S is a linearly independent set that is not already a basis for V, then S can be enlarged to a basis for V by inserting 
appropriate vectors into S. 


If S is a set of vectors that spans V but is not a basis for V, then S is a linearly dependent set. Thus some vector v in S is 
expressible as a linear combination of the other vectors in S. By the Plus/Minus Theorem (Theorem 5.4.4 b), we can remove v from 
S, and the resulting set S ! will still span V. If S 1 is linearly independent, then S f is a basis for V, and we are done. If S' is linearly 
dependent, then we can remove some appropriate vector from S ! to produce a set S ,f that still spans V. We can continue removing 
vectors in this way until we finally arrive at a set of vectors in S that is linearly independent and spans V. This subset of S is a basis 
for V. 




Suppose that dirn(F) = n- If S is a linearly independent set that is not already a basis for V, then S fails to span V, and 
there is some vector v in V that is not in span(S). By the Plus/Minus Theorem (Theorem 5.4.4c;), we can insert v into S, and the 
resulting set g 1 will still be linearly independent. If <$ f spans V, then is a basis for V, and we are finished. If jjfi does not span V, 
then we can insert an appropriate vector into S r to produce a set S !! that is still linearly independent. We can continue inserting 
vectors in this way until we reach a set with n linearly independent vectors in V. This set will be a basis for Fby Theorem 5.4.5. 


It can be proved (Exercise 30) that any subspace of a finite-dimensional vector space is finite-dimensional. We conclude this 
section with a theorem showing that the dimension of a subspace of a finite-dimensional vector space V cannot exceed the 
dimension of V itself and that the only way a subspace can have the same dimension as V is if the subspace is the entire vector 
space V. Figure 5.4.6 illustrates this idea in p}. In that figure, observe that successively larger subspaces increase in dimension. 



THEOREM 5.4.7 


If W is a subspace of a finite-dimensional vector space V, then dim(fF) < dirn(F),' moreover, {/dim (IT) = dim(F), then W= V- 


Since V is finite-dimensional, so is W by Exercise 30. Accordingly, suppose that s= (wj, W 2 , w„,j is a basis for VP. 
Either S is also a basis for V or it is not. If it is, then dim(JF) = dirn(F) = m- If it is not, then by Theorem 5.4.6/?, vectors can be 
added to the linearly independent set S to make it into a basis for V, so dun (IF) < dun(F) • Thus dim (IF) < dun(F) in all cases. If 
dim (IF) = dim(F), then S is a set of m linearly independent vectors in the m-dimensional vector space V ; hence S is a basis for V 
by Theorem 5.4.5. This implies that W=V (why?). 


Additional Proofs 


Assume that s= (vj, ..., v r ) is a linearly independent set of vectors in V, and v is a vector in V 

outside of span(5). To show that S ! = (vi, \' 2 ,..., v) is a linearly independent set, we must show that the only scalars that 

satisfy 


*ivi -I- k 2 v 2 + - + k y \’ r + jtj. + iv = 0 

are ki=k 2 = - = k r = ^ + i = 0. But we must have ^ +1 = o; otherwise, we could solve 11 for v as a linear 
combination of Vl , v 2 ,..., v,., contradicting the assumption that v is outside of span(S). Thus 11 simplifies 
to 
















^l v l + ^ 2 V 2 H-1- -v> = 0 


( 12 ) 


which, by the linear independence of {vi, v 2 ,\>}, implies that 

jti = k 2 = ■" = k r = 0 


Assume that s= {vj, v 2 ,v r } is a set of vectors in V, and to be specific, suppose that v r is a linear 
combination of y 1; y 2> Vj._p say 


= civ 2 + c 2V2 + - + c r _iT r _i 


(13) 


We want to show that if \> is removed from S, then the remaining set of vectors { V1 , v 2 , \>_i) still 
spans span(S); that is, we must show that every vector w in span(S) is expressible as a linear 
combination of {vi, V 2 ,\>_i) ■ But if w is in span(S), then w is expressible in the form 

W = *1 vi + *2V2 -I-1- *J— 1 V >-1 k r v r 


or, on substituting 13, 

w = jfcivi k 2 V2 H-h + £ r (civi 4 4 -c r - iv>_i) 


which expresses w as a linear combination of V1 , v 2 , v r -i- 


Exercise Set 5.4 



Click here for Just Ask! 


Explain why the following sets of vectors are not bases for the indicated vector spaces. (Solve this problem by inspection.) 

1 . 


(a) m = (1, 2), u 2 = (0, 3), u 3 = (2, 7) for R 2 

(b) m = ( - 1, 3, 2), u 2 = (6, 1, 1) for R 3 


( c ) pi = 1 4 x 4 x 2 , P 2 = x — 1 for P 2 


(d) 



'1 f 

B = 

1 

O-i 

O 

C = 

O 

i:-"i 

i_ 

D = 

'5 r 

E = 

'7 r 

A = 


2 3 


-1 4 


1 7 


4 2 


2 9 


for M'22 


2 . 


Which of the following sets of vectors are bases for R 1 


(a) (2, 1), (3, 0) 


(b) (4, 1), (-7, -8) 















(c) (0,0), (1,3) 


(d) (3, 9), (-4, -12) 

Which of the following sets of vectors are bases for 

(a) (1,0,0), (2, 2,0), (3, 3, 3) 

(b) (3, 1,-4), (2, 5, 6), (1,4, 8) 

(c) (2, 3, 1), (4, 1, 1), (0, -7, 1) 

(d) (1,6, 4), (2, 4,-1), (-1,2, 5) 

Which of the following sets of vectors are bases for p£ 

(a) l-3*4-2* 2 , 1 I * + 4* 2 , 1-7* 

(b) 4 I 6* 4- * 2 , - 1 4-4*4 2* 2 , 5 4 2*-* 2 

(c) 1 f * 4- x 2 , x 4 x 2 , x 2 

(d) -44*4- 3* 2 , 6 I- 5* I 2* 2 , 8 I 4x f x 2 
Show that the following set of vectors is a basis for M22- 


f3 6 ' 


1 

1 

0 

1 _ 


1 

0 

1 

CO 
_1 


1 o' 

1 

'-O 

1 

i:- - i 

_ 1 


-1 0 


_ — 12 — 4_ 


_ - 1 2_ 


Let Vbe the space spanned by Vl = C0S 2 X , v - 2 = sin 2 *’ v 3 = cos2*- 

(a) Show that S = {yj, V2, v 3 ) is not a basis for V. 

(b) Find a basis for V. 


Find the coordinate vector of w relative to the basis S= {uj, 112 } lor p 2 


(a) 111 = (1.0). u 2 = (0, 1); w= (3, -7) 



(b) 111 = (2, -4), u 2 = (3, 8); w=(l, 1) 


(c) ui = (1, 1), u 2 = (0. 2); w = (a, b ) 


Find the coordinate vector of w relative to the basis S = («], 112 } of g}. 

8. 

(a) ui = (1, -1), n 2 = (l, 1); w=(l,0) 

(b) m = (l, -1), 112 = (1,1); w = (0, 1) 

(c) ui = (1, -1), n 2 = (l, 1); w=(l, 1) 

Find the coordinate vector of v relative to the basis S= {vj, v 2 , v 3 ) • 

9. 

(a) v= (2, -1,3); vi = (1, 0, 0), v 2 (2, 2, 0), v 3 = (3, 3, 3) 


(b) v= (5, -12,3); vi = (1,2, 3), v 2 (-4, 5, 6 ), v 3 = (7, -8,9) 

Find the coordinate vector of p relative to the basis S= {pi, p 2 , P 3 } • 

(a) p =4 — 3x I x 2 ; pi = 1, p 2 = x, p 3 = x 2 

(b) p = 2 — x \- x 2 ; pi = 1 fi, p 2 = 1 4- x 2 , p 3 = x 4 x 2 


Find the coordinate vector of A relative to the basis S= {A\, A 2 , Aj, A 4 ) ■ 


2 0‘ 


' -1 r 


'1 r 


'0 o' 


'0 o' 

_ -1 3_ 


0 0_ 


_0 0_ 

^3 = 

_1 0_ 

-^4 = 

_0 1_ 


In Exercises 12-17 determine the dimension of and a basis for the solution space of the system. 

*1 I * 2-*3 =0 
— 2x\ — X 2 4 2 x 2 = 0 
— xi 4 x 3 = 0 


13 . 


3xi I *2 f *3 4 X 4 = 0 
5xi—X 2 I x 3 — X4=0 


x 1—4x2 4 3x3 — X 4=0 
2x1 — 8x24 6x3 — 2x4=0 



x i — 3 x 2 + X 2 = 0 
2x\ —6x2-\- 2x2 = 0 
3x\ — 9x2 + 3x3 = 0 

2x\ -{ X 2 I 3x3 = 0 

xi H 5x3 = 0 
X 2 4 13 = 0 

x \-y -I z = 0 
17 • 3x-f2j-2z = 0 
4x f 3j — z = 0 
6x I 5j +z = 0 

Determine bases for the following subspaces of p}. 

18. 

(a) the plane 3x — 2 y -4- 5 z = 0 

(b) the plane x —y = 0 


(c) the line x = 21 , y = —t,z = 4 t 

(d) all vectors of the form (a, b,c), where b = a 4 - c 


Determine the dimensions of the following subspaces of j? 4 . 

19. 

(a) all vectors of the form (a, b,c, 0) 

(b) all vectors of the form ( a , b, c, d), where d = a + b and c = a-b 

(c) all vectors of the form ( a , b, c, d), where a = b = c = d 


Determine the dimension of the subspace of p 3 consisting of all polynomials ao + a\x + ^x 2 + a, 2 X? f° r which arj 


Find a standard basis vector that can be added to the set (yj, V 2 } t0 produce a basis for p}. 
(a) vi = (-1,2, 3), v 2 = (l, -2,-2) 


(b) v 1 = (l, -1,0), v 2 = (3, 1, -2) 


Find standard basis vectors that can be added to the set ( Vli y 2 } to produce a basis for 
V1 = (1, -4, 2, -3), v 2 = (-3, 8, -4, 6) 


22 . 







Let (vj, y 2 , V3} be a basis for a vector space V. Show that (uy U 2,113} is also a basis, where uj = vi, 112 = vi + v 2 . and 
u 3 = vi f v 2 4 v 3 . 


24. 

(a) Show that for every positive integer n, one can find n -| 1 linearly independent vectors in p ( _ rxj, do ) • 

Hint Look for polynomials. 

(b) Use the result in part (a) to prove that F( — co , 00 ) is infinite-dimensional. 

(c) Prove that C( — do , 00 )> — 00 , 00 ), and C ' ( — do , 00 ) are infinite-dimensional vector spaces. 

Let S’ be a basis for an ^-dimensional vector space V. Show that if Vli y 2i y r form a linearly independent set of vectors in V, 
25* then the coordinate vectors (yj) (v 2 ) s ,(y r ) s form a linearly independent set in R n , and conversely. 

Using the notation from Exercise 25, show that if Vli y 2i y,. span V, then the coordinate vectors ( Vl ) (y 2 ) ( Vj .) span 

26- pF, and conversely. 

Find a basis for the subspace of p, spanned by the given vectors. 

27. 

(a) -I n- 2 x 2 , 3 + 3 x + 6 x 2 , 9 

(b) 1 1 - x, x 2 , - 2 4 2x 2 , - 3x 

(c) H x - 3x 2 , 2 4 2x - 6x 2 , 3 f 3x - 9x 2 

Hint Let S be the standard basis for _p 2 and work with the coordinate vectors relative to .S’; note Exercises 25 and 26. 

The accompanying figure shows a rectangular xy-coordinate system and an x'jy^-coordinate system with skewed axes. 

^ O t t 

Assuming that 1-unit scales are used on all the axes, find the x y -coordinates of the points whose xjy-coordinates are given. 

(a) (L 1) 

(b) (1,0) 

(c) (0, 1) 


(d) (a, b) 




Figure Ex-28 


The accompanying figure shows a rectangular ^-coordinate system determined by the unit basis vectors i and j and an x'y' 
29' -coordinate system determined by unit basis vectors uj and 112 - Find the x'^-coordinates of the points whose ^-coordinates 


are given. 

(a) 


(b) 

(1,0) 

(c) 

(0, 1) 

(d) 

(a, b) 



Figure Ex-29 


Prove: Any subspace of a finite-dimensional vector space is finite-dimensional. 

30. 


Discussion 

Discovery 


The basis that we gave for Mji in Example 6 consisted of noninvertible matrices. Do you think that 
31* there is a basis for M 22 consisting of invertible matrices? Justify your answer. 


32. 

(a) The vector space of all diagonal nxn matrices has dimension _ 

(b) The vector space of all symmetric nxn matrices has dimension 


(c) The vector space of all upper triangular nxn matrices has dimension 












33 . 


(a) For a 3 x 3 matrix A, explain in words why the set / 3 , A, A?, must be linearly 
dependent if the ten matrices are distinct. 


(b) State a corresponding result for an n x.n matrix A. 


State the two parts of Theorem 5.4.2 in contrapositive form. [See Exercise 34 of Section 1.4.] 

34. 


35. 

(a) The equation 7:i | X2 -\ _ V = 0 can be viewed as a linear system of one equation in n 

unknowns. Make a conjecture about the dimension of its solution space. 


(b) Confirm your conjecture by finding a basis. 


36. 

(a) Show that the set W of polynomials in p 2 such that ^(l)=oisa subspace of p 2 . 

(b) Make a conjecture about the dimension of W. 

(c) Confirm your conjecture by finding a basis for W. 


Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 



5.5 

ROW SPACE, COLUMN 
SPACE, AND NULLSPACE 


In this section we shall study three important vector spaces that are associated 
with matrices. Our work here will provide us with a deeper understanding of the 
relationships between the solutions of a linear system of equations and 
properties of its coefficient matrix. 


We begin with some definitions. 


DEFINITION 


For an m x n matrix 


the vectors 



"an 

ai2 ' 



A = 

«21 

a 22 ‘ 

" a 2n 



a m\ 

a m2 ‘ 

" a 7nn 


i’l = [an 

ai2 

fli H ] 

r 2 = [«21 

a 22 


r m = \. a m\ 

am2 



in R n formed from the rows of A are called the row vectors of A, and the vectors 



i 

—' CN 

1_ 


"ai2 ’ 
a 22 


^ 1h 

a 2 H 

ci = 


c 2 — 

a m 2 

» - c n~ 



in R m formed from the columns of A are called the column vectors of A. 


Row and Column Vectors in a 2 x 3 Matrix 


Let 


The row vectors of A are 


and the column vectors of A are 



ri = [2 1 0] and r 2 =[3 -1 4] 



"2" 


r 


"o" 

Cl = 

3 

c 2 — 

-i 

B 

o 

LO 

II 

4 


♦ 


The following definition defines three important vector spaces associated with a matrix. 


























DEFINITION 


If A is an m x n matrix, then the subspace of R n spanned by the row vectors of A is called the rowspace of A, and the subspace 
ofR m spanned by the column vectors of A is called the column space of A. The solution space of the homogeneous system of 
equations j4 k = o, which is a subspace of R n , is called the nullspace of A. 


In this section and the next we shall be concerned with the following two general questions: 


What relationships exist between the solutions of a linear system = h and the row space, column space, and nullspace of 
the coefficient matrix A1 


What relationships exist among the row space, column space, and nullspace of a matrix? 


To investigate the first of these questions, suppose that 


'an 

«12 • 




"*r 

«21 

«22 ■ 


and 

x = 

*2 


a m2 ‘ 






It follows from Formula 10 of Section 1.3 that if ci, C 2 ,c H denote the column vectors of A, then the product j4 k can be 
expressed as a linear combination of these column vectors with coefficients from x; that is, 

j4x — xi ci 4- X 2 C 2 4- - + x„c H 


Thus a linear system, yj* = fo, of m equations in n unknowns can be written as 

xici + x 2 C2 + - + x H c H = b 


( 2 ) 


from which we conclude that j 4\ = b is consistent if and only if b is expressible as a linear combination of the column vectors of A 
or, equivalently, if and only if b is in the column space of A. This yields the following theorem. 


THEOREM 5.5.1 


A system of linear equations j4x = b is consistent if and only ifb is in the column space of A. 


A Vector b in the Column Space of A 


Let j4x = 1) be the linear system 


1 

■"] 

m 

i 

i_ 


■*r 


f 

12-3 


x 2 

= 

-9 

2 1-2 


x 3 


-3 


Show that b is in the column space of A, and express b as a linear combination of the column vectors of A. 




















Solution 


Solving the system by Gaussian elimination yields (verify) 

*1 = 2 , *2 = - 1 , *3 = 3 

Since the system is consistent, b is in the column space of A. Moreover, from 2 and the solution obtained, it follows that 


-1 


3 


2 


1 

1 

— 

2 

+ 3 

-3 

= 

-9 

2 


1 


-2 


-3 


The next theorem establishes a fundamental relationship between the solutions of a nonhomogeneous linear system j4x = b and 
those of the corresponding homogeneous linear system = 0 with the same coefficient matrix. 


THEOREM 5.5.2 


Ifx o denotes any single solution of a consistent linear system ^x = fo, and if v \, x 2 < .... reform a basis for the nullspace of 
A — that is, the solution space of the homogeneous system j4x = 0 —then every solution of Ax = b can be expressed in the form 

x = x 0 + c 1 vi f C2V2 + - + c*v* 

and, conversely, for all choices of scalars c i, c 2 , ... , c k , the vector x in this formula is a solution of 
Ax = 1). 


Assume that xq is any fixed solution of ylx = b and that x is an arbitrary solution. 


Then 


Subtracting these equations yields 


= h and j4x = b 


j4x — j4xn = 0 or j4(x — xn)=0 


which shows that x — xo is a solution of the homogeneous system j4x = 0- Since vq, V 2 > • • -v k is a basis for the solution space of 
this system, we can express x — xo as a linear combination of these vectors, say 

x - xq = civi 4- c 2 v 2 4-1- CfcVfc 


Thus, 


x = x 0 4- civi 4 c 2 v 2 4- - 4- c k v k 


which proves the first part of the theorem. Conversely, for all choices of the scalars cj, q,..., c k in 3, we have 

j4x = j4(xq 4- civi 4- C2 V 2 4-fCFfc) 


or 

^x = j4x 0 4- c i (j4v i) 4- c-fAx-f) 4- 4- 

But xo is a solution of the nonhomogeneous system, and vj, v’ 2 ,..., are solutions of the homogeneous system, so the last 
equation implies that 

J 4x = b + 04-04- - 4- 0 = b 

which shows that x is a solution of ylx = b- 


General and Particular Solutions 


There is some terminology associated with Formula 3. The vector xq is called a particular solution of yJx = b- The expression 










xg + civi H C'j \’2 H-1- Ck v k i s called the general solution of j4x = lj, and the expression c jvj +. C2\'2 H-1- £k v k * s called the 

general solution of j4x = 0- With this terminology, Formula 3 states that the general solution of Ax = b is the sum of any particular 
solution of Ax = b and the general solution of Ax = 0- 

For linear systems with two or three unknowns, Theorem 5.5.2 has a nice geometric interpretation in pf and gf. For example, 
consider the case where j4x = 0 and j4x = b are linear systems with two unknowns. The solutions of ^x = o form a subspace of p } 
and hence constitute a line through the origin, the origin only, or all of p A . From Theorem 5.5.2, the solutions of j4 x = b can be 
obtained by adding any particular solution of Jx = lj, say xo, to the solutions of j4 x = 0- Assuming that xo is positioned with its 
initial point at the origin, this has the geometric effect of translating the solution space of ^x = 0, so that the point at the origin is 
moved to the tip of xo (Figure 5.5.1). This means that the solution vectors of ^x = b form a line through the tip of xo, the point at 
the tip of xo, or all of pf. (Can you visualize the last case?) Similarly, for linear systems with three unknowns, the solutions of 
j4x = b constitute a plane through the tip of any particular solution xo, a line through the tip of xo, the point at the tip of xo, or all of 
R 3 - 




Ui) (M 


Figure 5.5.1 

Adding xq to each vector x in the solution space of ^x = 0 translates the solution space. 


General Solution of a Linear System = b 


In Example 4 of Section 1.2 we solved the nonhomogeneous linear system 

xi+ 3 x 2 —2^3 +2x5 = 0 

2 xi+6x2 — 5 x 3 —2x4+ 4 x 5 — 3 xg = — 1 
5 x 3 F 10x4 + 15xg —5 

2xi+6x2 +8x4+4x5+18x6 =6 


and obtained 


xi = — 3r — 4s — 2t, X 2 =r. 


X 3 = — 2s, 


X 4 = s. 


X 5 = t. 


This result can be written in vector form as 




*2 


*3 


x 4 




*6 



_ 


— 2r — As —2t 
r 

— 2s 
s 

t 

1 

3 




which is the general solution of 4. The vector xq in 5 is a particular solution of 4; the linear combination x in 5 is the general 


(4) 


( 5 ) 























solution of the homogeneous system 


(verify). 


*1 h 3 x 2 — 2 x 2 + 2 x$ =0 

2x\ 6 x 2 — 5*3 — 2 x 4 + 4 x 5 — 3*6 = 0 

5*3 4 10*4 +15*6 = 0 

2*i 1-6*2 +8*4+4*5 + 18*6 = 0 


Bases for Row Spaces, Column Spaces, and Nullspaces 

We first developed elementary row operations for the purpose of solving linear systems, and we know from that work that 
performing an elementary row operation on an augmented matrix does not change the solution set of the corresponding linear 
system. It follows that applying an elementary row operation to a matrix A does not change the solution set of the corresponding 
linear system = Q, or, stated another way, it does not change the nullspace of A. Thus we have the following theorem. 


THEOREM 5.5.3 


Elementary row operations do not change the nullspace of a matrix. 


EXAMPLE 4 Basis for Nullspace 


Find a basis for the nullspace of 


2 



0 


2 -1 

-1 2 

1 -2 

0 1 


0 1 

-3 1 

0 -1 
1 1 


Solution 

The nullspace of A is the solution space of the homogeneous system 

2*i f 2*2 — *3 +*5 = 0 

— *1 —*2+2*3 — 3*44*5 = 0 

*1 +*2—2*3 — *5 = 0 

*3 + *4 + *5 = 0 

In Example 10 of Section 5.4 we showed that the vectors 


’ -1" 


’ - f 

1 


0 

0 

and V 2 = 

-1 

0 


0 

0 


1 


form a basis for this space. 


The following theorem is a companion to Theorem 5.5.3. 











THEOREM 5.5.4 


Elementary row operations do not change the row space of a matrix. 


Suppose that the row vectors of a matrix A are i’i, i’ 2 » • • • » r m , and let B be obtained from A by performing an elementary 
row operation. We shall show that every vector in the row space of B is also in the row space of A and that, conversely, every 
vector in the row space of A is in the row space of B. We can then conclude that A and B have the same row space. 

Consider the possibilities: If the row operation is a row interchange, then B and A have the same row vectors and consequently 
have the same row space. If the row operation is multiplication of a row by a nonzero scalar or the addition of a multiple of one 
row to another, then the row vectors r ', . .., v ! m of B are linear combinations of iq, i’ 2 , .. .,r m ; thus they lie in the row space of A. 

Since a vector space is closed under addition and scalar multiplication, all linear combinations °f l’j, r^, • • r (,, will also lie in the 
row space of A. Therefore, each vector in the row space of B is in the row space of A. 

Since B is obtained from A by performing a row operation, A can be obtained from B by performing the inverse operation (Section 
1.5). Thus the argument above shows that the row space of A is contained in the row space of B. 


In light of Theorems Theorem 5.5.3 and Theorem 5.5.4, one might anticipate that elementary row operations should not change the 
column space of a matrix. However, this is not so— elementary row operations can change the column space. For example, 
consider the matrix 


The second column is a scalar multiple of the first, so the column space of A consists of all scalar multiples of the first column 
vector. However, if we add -2 times the first row of A to the second row, we obtain 


Here again the second column is a scalar multiple of the first, so the column space of B consists of all scalar multiples of the first 
column vector. This is not the same as the column space of A. 


Although elementary row operations can change the column space of a matrix, we shall show that whatever relationships of linear 
independence or linear dependence exist among the column vectors prior to a row operation will also hold for the corresponding 
columns of the matrix that results from that operation. To make this more precise, suppose a matrix B results from performing an 
elementary row operation on an m x n matrix A. By Theorem 5.5.3, the two homogeneous linear systems 

Ax = 0 and Bx = 0 

have the same solution set. Thus the first system has a nontrivial solution if and only if the same is true of the second. But if the 
column vectors of A and B, respectively, are 

ci, C 2 ,c H and c„ 

then from 2 the two systems can be rewritten as 

*1*1 +*2C2 + —+ * M c« = 0 
and 

*ic'l +* 2 C2 + =0 

Thus 6 has a nontrivial solution for xq, X 2 > .. .,x H if and only if the same is true of 7. This implies that the column vectors of A are 
linearly independent if and only if the same is true of B. Although we shall omit the proof, this conclusion also applies to any 
subset of the column vectors. Thus we have the following result. 







THEOREM 5.5.5 


If A and B are row equivalent matrices, then 

(a) A given set of column vectors of A is linearly independent if and only if the corresponding column vectors ofB are 
linearly independent. 

(b) A given set of column vectors of A forms a basis for the column space of A if and only if the corresponding column 
vectors of B form a basis for the column space ofB. 


The following theorem makes it possible to find bases for the row and column spaces of a matrix in row-echelon form by 
inspection. 


THEOREM 5.5.6 


If a matrix R is in row-echelon form, then the row vectors with the leading I's (the nonzero row vectors) form a basis for the 
row space ofR, and the column vectors with the leading 1' s of the row vectors form a basis for the column space ofR. 


Since this result is virtually self-evident when one looks at numerical examples, we shall omit the proof; the proof involves little 
more than an analysis of the positions of the 0's and l's of R. 


IRXA Bases for Row and Column Spaces 


The matrix 

‘l -2 5 0 3‘ 

0 13 0 0 

_ 0 0 0 1 0 

0 0 0 0 0 

is in row-echelon form. From Theorem 5.5.6, the vectors 

i’i = [1 -2 5 0 3] 

r 2 — [0 1 3 0 0] 

r 3 — [ 0 0 0 1 0 ] 

form a basis for the row space of R, and the vectors 



T 


' -2‘ 


‘o' 


0 


1 


0 

Cl = 

0 

c 2 = 

0 

c 4 

1 


0 


0 


0 


form a basis for the column space of R. 


♦ 















Bases for Row and Column Spaces 


Find bases for the row and column spaces of 


1 

-3 

4 

-2 

5 

4 

2 

-6 

9 

- 1 

8 

2 

2 

-6 

9 

- 1 

9 

7 

1 

3 

-4 

2 

-5 

-4 


Solution 


Since elementary row operations do not change the row space of a matrix, we can find a basis for the row space of A by finding a 
basis for the row space of any row-echelon form of A. Reducing A to row-echelon form, we obtain (verify) 

'l -3 4 -2 5 4' 

„ 0 01 3-2-6 

A 0 0 0 0 1 5 

0 0 0 0 0 0 

By Theorem 5.5.6, the nonzero row vectors of R form a basis for the row space of R and hence form a basis for the row space of A. 
These basis vectors are 

i’l = [1 -3 4 -2 5 4] 

r 2 = [0 0 1 3-2-6] 

r 3 = [0 0 0 0 1 5] 


Keeping in mind that A and R may have different column spaces, we cannot find a basis for the column space of A directly from 
the column vectors of R. However, it follows from Theorem 5.5.5 b that if we can find a set of column vectors of R that forms a 
basis for the column space of R, then the corresponding column vectors of A will form a basis for the column space of A. 


The first, third, and fifth columns of R contain the leading 1 's of the row vectors, so 


T 


" 4 " 


5 " 

0 

J 

1 

J 

-2 

0 

- 

0 

c 5 ~ 

1 

0 


0 


0 


form a basis for the column space of R; thus the corresponding column vectors of A —namely, 


f 


4 ' 


5' 

2 


9 


8 

2 

c 3 = 

9 

= 

9 

- 1 


-4 


-5 


form a basis for the column space of A. 


♦ 


Basis for a Vector Space Using Row Operations 


Find a basis for the space spanned by the vectors 

vi = (1. - 2, 0, 0, 3), v 2 = (2, - 5, - 3, - 2, 6), 

V 3 = (0.5, 15, 10,0), v 4 = (2, 6, 18,8,6) 





















Solution 


Except for a variation in notation, the space spanned by these vectors is the row space of the matrix 

'1 -2 0 0 3' 

2 -5 -3 -2 6 

0 5 15 10 0 

2 6 18 8 6 

Reducing this matrix to row-echelon form, we obtain 

'l -2 0 0 3' 

0 13 2 0 

0 0 110 
0 0 0 0 0 

The nonzero row vectors in this matrix are 

wi = (l. -2,0,0, 3), w '2 = (0, 1, 3, 2, 0), w 3 = (0,0, 1,1,0) 

These vectors form a basis for the row space and consequently form a basis for the subspace of p} spanned by vj, V 2 > v 3 , and V 4 . 


Observe that in Example 6 the basis vectors obtained for the column space of A consisted of column vectors of A, but the basis 
vectors obtained for the row space of A were not all row vectors of A. The following example illustrates a procedure for finding a 
basis for the row space of a matrix A that consists entirely of row vectors of A. 


EXAMPLE 8 Basis for the Row Space of a Matrix 


Find a basis for the row space of 


consisting entirely of row vectors from A. 


1 -2 



2 6 


0 0 3 

-3 -2 6 

15 10 0 

18 8 6 


Solution 


We will transpose A, thereby converting the row space of A into the column space of then we will use the method of Example 
6 to find a basis for the column space of and then we will transpose again to convert column vectors back to row vectors. 
Transposing A yields 


A T 


1 2 0 2 

2-556 
0 -3 15 18 

0 -2 10 8 

3 6 0 6 


Reducing this matrix to row-echelon form yields 

’l2 0 2 

0 1 -5 -10 

0 0 0 1 

0 0 0 0 

0 0 0 0 


The first, second, and fourth columns contain the leading l's, so the corresponding column vectors in form a basis for the 














column space of A these are 


f 


2 " 


’ 2 " 

-2 


— 5 


6 

0 

c 2 = 

-3 

II 

’=T 

U 

3 

18 

0 


-2 


8 

3 


6 


6 


Transposing again and adjusting the notation appropriately yields the basis vectors 

i’! = [l -2 0 0 3], i’2=[2 -5 -3 -2 6], 


and 

r 4 = [2 6 18 8 6] 


for the row space of A. 


♦ 


We know from Theorem 5.5.5 that elementary row operations do not alter relationships of linear independence and linear 
dependence among the column vectors; however, Formulas 6 and 7 imply an even deeper result. Because these formulas actually 
have the same scalar coefficients x\, * 3 , ... , x n , it follows that elementary row operations do not alter the formulas (linear 
combinations) that relate linearly dependent column vectors. We omit the formal proof. 


Basis and Linear Combinations 


(a) Find a subset of the vectors 


vi = (1, - 2, 0, 3), v 2 = (2, - 5, -3,6), 
v 3 = (0, 1,3,0), v 4 = (2, - 1,4, -7), VJ = (5, -8, 1,2) 

that forms a basis for the space spanned by these vectors. 


(b) Express each vector not in the basis as a linear combination of the basis vectors. 


Solution (a) 


We begin by constructing a matrix that has vj. 

V2. •••> 

v -5 as its column vectors: 


! 

2 

{) 

2 5 


~ 

—:5 

1 

-1 “H 


0 

—3 

J 

4 1 


3 

6 

0 

-7 2 


t 

+ 


Vi 

V; 

Y\ 

V4 v 5 


The first part of our problem can be solved by finding a basis for the column space of this matrix. Reducing the matrix to reduced 
row-echelon form and denoting the column vectors of the resulting matrix by wj, W 2 , w 3 , W 4 , and yields 












10 2 0 1 

0 I “1 0 1 

0 0 0 1 I 

0 0 0 D 0 

t T t T T 

W i ivi w.i w, s w 5 


The leading l's occur in columns 1, 2, and 4, so by Theorem 5.5.6, 

{vr h w 2 , w 4 } 

is a basis for the column space of 9, and consequently, 

{vi, V 2 , v 4 ) 

is a basis for the column space of 9. 

Solution (b) 

We shall start by expressing W 3 and as linear combinations of the basis vectors wj, w 2 , w 4 . The simplest way of doing this is to 
express W 3 and w; in terms of basis vectors with smaller subscripts. Thus we shall express W 3 as a linear combination of w 4 and w 2 
, and we shall express w; as a linear combination of wj, w 2 , and w 4 . By inspection of 9, these linear combinations are 

W 3 = 2 wi - w 2 
w '5 = wi +■ w 2 4 - w 4 

We call these the dependency equations. The corresponding relationships in 8 are 

V 3 = 2 vi - v 2 
VJ = VI + v 2 + v 4 

♦ 


The procedure illustrated in the preceding example is sufficiently important that we shall summarize the steps: 


Given a set of vectors S= (vj, v 2 ,..., v*} in R n , the following procedure produces a subset of these vectors that forms a basis 
for span(.S’) and expresses those vectors of S that are not in the basis as linear combinations of the basis vectors. 


Form the matrix A having vj, v 2 , ..., as its column vectors. 

Reduce the matrix A to its reduced row-echelon form R, and let wj, w 2 , ..., w& be the column vectors of R. 

Step 3 Identify the columns that contain the leading l's in R. The corresponding column vectors of A are the basis vectors for 
span (5). 

Step 4 Express each column vector of R that does not contain a leading 1 as a linear combination of preceding column vectors 
that do contain leading l's. (You will be able to do this by inspection.) This yields a set of dependency equations 
involving the column vectors of R. The corresponding equations for the column vectors of A express the vectors that 
are not in the basis as linear combinations of the basis vectors. 


Exercise Set 5.5 


Click here for Just Ask! 






List the row vectors and column vectors of the matrix 


2-10 1 
3 5 7 -1 

1 4 2 7 


Express the product as a linear combination of the column vectors of A. 

2 . 


(a) 


(b) 



2 3' 

'f 

— 

1 4_ 

_2_ 

'4 

0 

- 1 

3 

6 

2 

0 

- 1 

4 


-2 

3 

5 


(c) 



(d) 


2 1 
6 3 


3 

0 

-5 


3. 


Determine whether b is in the column space of A, and if so, express b as a linear combination of the column vectors of 


(a) A = 

'1 3' 

; b = 

' -2' 


_4 — 6_ 


10_ 


(b) 

'1 1 2' 


' - r 

A = 

1 0 1 

;b = 

0 


2 1 3 


2 


(c) 


(d) 



'i -i r 


5' 


A = 

9 3 1 

;b = 

1 



1 1 1 


- 1 



1-1 

f 


'2' 

A = 

1 1 

- 1 

; b = 

0 


- 1 - 1 

1 


0 



'1201' 


'4' 

0 12 1 

;b = 

3 

12 13 

5 

0 12 2 


7 


Suppose that ;q = _ 1 , * 2 = 2> *3 = 4> *4 = — 3 is a solution of a nonhomogeneous linear system ^ = b and that the solution 
4* set of the homogeneous system ^x = 0 is given by the formulas 

* 1 = — 3r I 4s, X 2 = r — s, *3 = r, *4 = s 


(a) Find the vector form of the general solution of ^4x = 0- 


(b) Find the vector form of the general solution of ^x = b- 


5. 


Find the vector form of the general solution of the given linear system = b; then use that result to find the vector form of the 
general solution of = 0 - 


(a) 

*1 - 

3 * 2=1 




2 xi — 

6*2 = 2 



(b) 

xy + 

x 2 4 - 2 x 3 

= 5 



*1 

+ *3= - 

-2 



2 xi + 

X2 + 3*3 

= 3 


(c) 

*1 ■ 

- 2*2 4 *3 

f 2*4 = 

- 1 


2 x\ - 

- 4*2 V 2*3 

I 4*4 = 

-2 


“*1 

[-0 

* 

to 

1 

LO 

- 2*4 

= 1 


3 *i - 

- 6*2 V 3*3 

I 6*4 = 

-3 

(d) 

*1 

H 2*2 — 3*3 

-f *4 

—< 


— 2 xi 

f *2 4 2*3 

1 *4 = 

— 


-xi 

4 3*2 — *3 

4 2*4 

=! 


4 xi 

- 7*2 

— 5*4 = 

—: 


Find a basis for the nullspace of A. 


(a) 1-1 3 

A= 5 -4 -4 
7-6 2 


6. 



(b) [2 0 -l' 

A= 4 0 -2 

0 0 0 

(c) [ 1 4 5 2' 

A= 2 13 0 

-13 2 2 

1 4 5 6 9 

3-2 1 4-1 

_1 o -1 -2 -1 


(e) 1 -3 2 2 1 

0360-3 
A= 2-3-2 4 4 

3 -6 0 6 5 

-2 9 2-4 -5 


In each part, a matrix in row-echelon form is given. By inspection, find bases for the row and column spaces of A. 

7. 

(a) [1 0 2' 

0 0 1 
0 0 0 

(b) [l -3 0 0 

0 10 0 

0 0 0 0 

0 0 0 0 


(c) 1 2 4 

0 1-3 0 

0 0 1-3 

0 0 0 1 

0 0 0 0 

(d) [1 

0 1 

0 0 

0 0 


For the matrices in Exercise 6, find a basis for the row space of A by reducing the matrix to row-echelon form. 




8. 







For the matrices in Exercise 6, find a basis for the column space of A. 

9. 

For the matrices in Exercise 6, find a basis for the row space of A consisting entirely of row vectors of A. 

10 . 

Find a basis for the subspace of [A spanned by the given vectors. 

11 . 


(a) (1, 1, -4, -3), (2, 0, 2, -2), (2, -1, 3, 2) 


(b) (-1, 1,-2,0), (3, 3, 6,0), (9, 0,0, 3) 


(c) (1, 1, 0, 0), (0, 0, 1, 1), (-2, 0, 2, 2), (0, -3, 0, 3) 


Find a subset of the vectors that forms a basis for the space spanned by the vectors; then express each vector that is not in the 
12. basis as a linear combination of the basis vectors. 


(a) vi = (1, 0, 1, 1), v 2 = ( - 3, 3, 7, 1), v 3 = ( - 1. 3, 9, 3), v 4 = ( - 5, 3, 5, - 1) 


(b) v 1 = (l, — 2, 0, 3). V 2 = (2, —4, 0, 6). v 3 = ( — 1, 1, 2, 0)> v 4 = (0, -1,2,3) 


(c) v! = (1, — 1, 5, 2)> V 2 = ( — 2, 3, 1, 0). v 3 = (4, - 5, 9, 4), v 4 = (0, 4, 2, - 3), v 3 = ( -7, 18, 2, -8) 


Prove that the row vectors of an n x n invertible matrix A form a basis for R n . 

13. 


14. 

(a) Let 

'0 1 O' 

A= 10 0 
0 0 0 

and consider a rectangular xyz-coordinate system in 3-space. Show that the nullspace of A 
consists of all points on the z-axis and that the column space consists of all points in the xy 
-plane (see the accompanying figure). 

(b) Find a 3 x 3 matrix whose nullspace is the v-axis and whose column space is the yz-plane. 





NtilUpiKC o VA 



Figure Ex-14 


Find a 3 x 3 matrix whose nullspace is 

15. 

(a) a point 

(b) aline 

(c) a plane 


Discussion 

Discovery 


Indicate whether each statement is always true or sometimes false. Justify your answer by giving a 
16. logical argument or a counterexample. 


(a) If E is an elementary matrix, then A and EA must have the same nullspace. 


(b) If E is an elementary matrix, then A and EA must have the same row space. 


(c) If E is an elementary matrix, then A and EA must have the same column space. 


(d) If = 1 ) does not have any solutions, then b is not in the column space of A. 


(e) The row space and nullspace of an invertible matrix are the same. 


17. 

(a) Find all 2 x 2 matrices whose nullspace is the line 3x — 5y = 0- 


(b) Sketch the nullspaces of the following matrices: 

0 0 
0 0 


A = 


1 4 
0 5 


B = 


1 0 
0 5 


C = 


6 2 
3 1 


D = 


18. 


The equation * j ^ 4 - *3 = 1 can be viewed as a linear system of one equation in three unknowns 
Express its general solution as a particular solution plus the general solution of the corresponding 














homogeneous system. [Write the vectors in column form.] 


Suppose that A and B are n x n matrices and A is invertible. Invent and prove a theorem that 
19. describes how the row spaces of AB and B are related. 
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5 £ In the preceding section we investigated the relationships between systems 

■O of linear equations and the row space, column space, and nullspace of the 

RANK AND NULLITY 

between the dimensions of the row space, column space, and nullspace of a 
matrix and its transpose. The results we will obtain are fundamental and will 
provide deeper insights into linear systems and linear transformations. 


Four Fundamental Matrix Spaces 

If we consider a matrix A and its transpose jf? together, then there are six vector spaces of interest: 

row space of A row space of jfT 

column space of A column space of a t 


nullspace of A nullspace of T 

However, transposing a matrix converts row vectors into column vectors and column vectors into row vectors, so except for a 
difference in notation, the row space of a t is the same as the column space of A, and the column space of a t is the same as the 
row space of A. This leaves four vector spaces of interest: 

row space of A column space of A 
nullspace of A nullspace of a T 

These are known as the fundamental matrix spaces associated with A. If A is an m x n matrix, then the row space of A and the 
nullspace of A are subspaces of Rf, and the column space of A and the nullspace of a j are subspaces of R m . Our primary goal 
in this section is to establish relationships between the dimensions of these four vector spaces. 


Row and Column Spaces Have Equal Dimensions 


In Example 6 of Section 5.5, we found that the row and column spaces of the matrix 


1- 3 4-2 5 4 

2- 6 9-1 8 2 

J 2-6 9-1 9 7 

-1 3-4 2 -5 -4 

each have three basis vectors; that is, both are three-dimensional. It is not accidental that these dimensions are the same; it is a 
consequence of the following general result. 


THEOREM 5.6.1 


If A is any matrix, then the row space and column space of A have the same dimension. 


Proof Let R be any row-echelon form of A. It follows from Theorem 5.5.4 that 


dun (row space of j 4) = dun(row space of^?) 






and it follows from Theorem 5.5.5 b that 

dim(column space of A) = dim(column space ofi?) 

Thus the proof will be complete if we can show that the row space and column space of R have the 
same dimension. But the dimension of the row space of R is the number of nonzero rows, and the 
dimension of the column space of R is the number of columns that contain leading l's (Theorem 
5.5.6). However, the nonzero rows are precisely the rows in which the leading l's occur, so the 
number of leading l's and the number of nonzero rows are the same. This shows that the row space 
and column space of R have the same dimension. 


The dimensions of the row space, column space, and nullspace of a matrix are such important numbers that there is some 
notation and terminology associated with them. 


DEFINITION 


The common dimension of the row space and column space of a matrix A is called the rank of A and is denoted by rank(A); 
the dimension of the nullspace of A is called the nullity of A and is denoted by nullity(A). 


Rank and Nullity of a 4 x 6 Matrix 


Find the rank and nullity of the matrix 

'-1 20 4 5-3 

3- 72 0 1 4 

J 2-52 4 6 1 

4- 92-4-4 7 


Solution 

The reduced row-echelon form of A is 

'l 0 -4 -28 -37 13 

0 1 -2 -12 -16 5 

0 0 0 0 0 0 

0 0 0 0 0 0 


( 1 ) 


(verify). Since there are two nonzero rows (or, equivalently, two leading l's), the row space and column space are both 
two-dimensional, so rank (A) = 2- To find the nullity of A, we must find the dimension of the solution space of the linear 
system Ax = 0- This system can be solved by reducing the augmented matrix to reduced row-echelon form. The resulting 
matrix will be identical to 1 , except that it will have an additional last column of zeros, and the corresponding system of 
equations will be 

x 1 — 4 x 3 — 28 x 4 — 37 x 5 + 13x6 = 0 
X2 — 2x3 — 12x4 — 16x5 + 5 x 6 = 0 

or, on solving for the leading variables, 

xi = 4 x 3 + 28 x 4 + 37x5 — 13*6 
X 2 = 2 x 3 + 12 x 4 + 16 x 5 -5x6 


It follows that the general solution of the system is 













or, equivalently, 


x\=4r-\- 28s4 lit — 13 u 

X 2 = 2r 4 12s 4 1 6t — 5u 

*3 = r 

*4 = s 

x$ = t 

x^—u 


"*l" 


4 


28 


37 


- 13 

*2 


2 


12 


16 


— 5 

*3 


1 


0 


0 

+ u 

0 

*4 

= r 

0 

+ s 

1 

+ t 

0 

0 



0 


0 


1 


0 

x 6 


0 


0 


0 


1 


Because the four vectors on the right side of 3 form a basis for the solution space, nullity ( 41 ) = 4 . 


(3) 


♦ 


The following theorem states that a matrix and its transpose have the same rank. 


THEOREM 5.6.2 


T 

If A is any matrix, then rank (41) = rank (41 ). 


Proof 


T T 

rank(A) = dim (row space of A) = dim(column space of A ) = rank(A ) . 
The following theorem establishes an important relationship between the rank and nullity of a matrix. 

THEOREM 5.6.3 


Dimension Theorem for Matrices 



If A is a matrix with n columns, then 

rank (41) 4 nullity (41) = n 

(4) 


Proof Since A has n columns, the homogeneous linear system Ax = 0 has n unknowns (variables). These fall into two 
categories: the leading variables and the free variables. Thus 

number of leading 
variables 


+ 


number of free 
variables 


= a 

















But the number of leading variables is the same as the number of leading l's in the reduced 
row-echelon form of A, and this is the rank of A. Thus 


rank (.4) | 


number of free 
variables 


The number of free variables is equal to the nullity of A. This is so because the nullity of A is the 
dimension of the solution space of Ax = 0, which is the same as the number of parameters in the 
general solution [see 3, for example], which is the same as the number of free variables. Thus 

rank(y4) 4 nuUity(j4) =n 


The proof of the preceding theorem contains two results that are of importance in their own right. 


THEOREM 5.6.4 


If A is an mxn matrix, then 

(a) rank ( A) = the number of leading variables in the solution of Ax = 0- 


(b) nullity — the number of parameters in the general solution of Ax — 0- 


EXAMPLE 2 The Sum of Rank and Nullity 


The matrix 

'-1 2 0 4 5 -3' 

3 — 7 2 0 1 4 

J 2 -5 2 4 6 1 

4-92-4-4 7 

has 6 columns, so 

rank(b4) 4 nullity (j4) = 6 

This is consistent with Example 1, where we showed that 

rank(j4) = 2 and nullity(j4) =4 


♦ 


Number of Parameters in a General Solution 


Find the number of parameters in the general solution of = 0 if A is a 5 x 7 matrix of rank 3. 


Solution 


From 4, 











Thus there are four parameters. 


nuUity(A) = n — rank(A) =7 — 3 = 4 


♦ 

Suppose now that A is an m x n matrix of rank r; it follows from Theorem 5.6.2 that T is an n x m matrix of rank r. Applying 
Theorem 5.6.3 to A and jf? yields 

T 

nullity (A)=tt — r, nullity (j4 ) = m — r 

from which we deduce the following table relating the dimensions of the four fundamental spaces of an m x h matrix A of rank 
r. 


Fundamental Space 

Dimension 

Row space of A 

r 

Column space of A 

r 

Nullspace of A 

n — r 

Nullspace of a t 

m — r 


Applications of Rank 

The advent of the Internet has stimulated research on finding efficient methods for transmitting large amounts of digital 
data over communications lines with limited bandwidth. Digital data is commonly stored in matrix form, and many 
techniques for improving transmission speed use the rank of a matrix in some way. Rank plays a role because it measures 
the “redundancy” in a matrix in the sense that if A is an m x n matrix of rank k, then n — k of the column vectors and m — k 
of the row vectors can be expressed in terms ofk linarly independent column or row vectors. The essential idea in many 
data compression schemes is to approximate the original data set by a data set with smaller rank that conveys nearly the 
same information, then eliminate redundant vectors in the approximating set to speed up the transmission time. 


Maximum Value for Rank 

If A is an m x « matrix, then the row vectors lie in R n and the column vectors lie in R m . This implies that the row space of A is 
at most »-dimensional and that the column space is at most m-dimensional. Since the row and column spaces have the same 
dimension (the rank of A), we must conclude that if m * then the rank of A is at most the smaller of the values of m and n. 
We denote this by writing 


rank(A) < mm(ra,«) 

where min(m, n) denotes the smaller of the numbers m and n if m * n or denotes their common value if m = n- 


(5) 


Maximum Value of Rank for a 7 x 4 Matrix 


If A is a 7 x 4 matrix, then the rank of A is at most 4, and consequently, the seven row vectors must be linearly dependent. If A 
is a 4 x 7 matrix, then again the rank of A is at most 4, and consequently, the seven column vectors must be linearly dependent. 









Linear Systems of m Equations in n Unknowns 


In earlier sections we obtained a wide range of theorems concerning linear systems of n equations in n unknowns. (See 
Theorem 4.3.4.) We shall now turn our attention to linear systems of m equations in n unknowns in which m and n need not be 
the same. 

The following theorem specifies conditions under which a linear system of m equations in n unknowns is guaranteed to be 
consistent. 


THEOREM 5.6.5 

The Consistency Theorem 

IfAx = h is a linear system of m equations in n unknowns, then the following are equivalent. 

( a ) Ax = b is consistent. 

(b) b is in the column space of A. 

(c) The coefficient matrix A and the augmented matrix [j4 | b] have the same rank. 


It suffices to prove the two equivalences (a) ^ (b) and (£) ^ (c ), since it will then follow as a matter of logic that 
(a) ** (c)- 

See Theorem 5.5.1. 

We will show that if b is in the column space of A, then the column spaces of A and [A | b] are actually the same, 
from which it will follow that these two matrices have the same rank. 

By definition, the column space of a matrix is the space spanned by its column vectors, so the column spaces of A and 
[A | b ] can be expressed as 

span{ci, ( 2 , c H ) and span(ci, C 2 >---> b) 

respectively. If b is in the column space of A, then each vector in the set (ci, f2> ---> f h> b} ' s a l> near combination of the 
vectors in ( Cli c„) and conversely (why?). Thus, from Theorem 5.2.4, the column spaces of A and [A | b] are the 
same. 

Assume that A and [A | b] have the same rank r. By Theorem 5.4.6a, there is some subset of the column vectors of 

A that forms a basis for the column space of A. Suppose that those column vectors are 

,/ J J 
C 1 > ( 2 ’ ■■■’ 

These r basis vectors also belong to the r-dimensional column space of [A | b ]; hence they also form a basis for the column 
space of [A | b] by Theorem 5.4.6a. This means that b is expressible as a linear combination of <2 , c and 
consequently b lies in the column space of A. 


It is not hard to visualize why this theorem is true if one views the rank of a matrix as the number of nonzero rows in its 



reduced row-echelon form. For example, the augmented matrix for the system 


— 2*2 — 3*3 4 - 2*4 = — 4 

i 

-2 

-3 

2 

- 4 ’ 

3 *i f 7*2 — *3 H *4 = — 3 

-3 

7 

- 1 

1 

-3 

is 

2 *i — 5*2 4 - 4*3 — 3*4 = 7 

2 

-5 

4 

-3 

7 

3 *i -h 6*2 4 - 9*3 — 6*4 = — 1 

-3 

6 

9 

-6 

- 1 


which has the following reduced row-echelon form (verify): 


'l 

0 

-23 

16 

0’ 

0 

1 

- 10 

7 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

0 


We see from the third row in this matrix that the system is inconsistent. However, it is also because of this row that the reduced 
row-echelon form of the augmented matrix has fewer zero rows than the reduced row-echelon form of the coefficient matrix. 
This forces the coefficient matrix and the augmented matrix for the system to have different ranks. 

The Consistency Theorem is concerned with conditions under which a linear system Ax — h is consistent for a specific vector b. 
The following theorem is concerned with conditions under which a linear system is consistent for all possible choices of b. 


THEOREM 5.6.6 


If Ax —h is a linear system of m equations in n unknowns , then the following are equivalent . 

( a ) Ax = b is consistent for every m x 1 matrix b. 

(b) The column vectors of A span 
(C) rank(A) = m- 


It suffices to prove the two equivalences (a) (b) and (a) ^ (c), since it will then follow as a matter of logic that 

(*) CO- 

From Formula 2 of Section 5.5, the system — b can be expressed as 

*1<T f *2 f 2 H-hx H c H =1) 

from which we can conclude that Ax = b is consistent for every matrix b if and only if every such b is expressible as a 
linear combination of the column vectors cj, C2>- • •> o H , or, equivalently, if and only if these column vectors span R m . 

From the assumption that Ax = b is consistent for every m x 1 matrix b, and from parts (a) and (b) of the Consistency 
Theorem (Theorem 5.6.5), it follows that every vector b in R m lies in the column space of A; that is, the column space of A is 
all of R m . Thus rank(A) = dim(i? m ) = m. 

From the assumption that rank(A) = m, it follows that the column space of A is a subspace of R m of dimension m and 
hence must be all of R m by Theorem 5.4.7. It now follows from parts (a) and (b) of the Consistency Theorem (Theorem 5.6.5) 
that Ax — h is consistent for every vector b in R m , since every such b is in the column space of A. 


A linear system with more equations than unknowns is called an overdetermined linear system. If Ax = b is an overdetermined 









linear system of m equations in n unknowns (so that m > «), then the column vectors of A cannot span R m ; it follows from the 
last theorem that for a fixed my.n matrix A with m > «, the overdetermined linear system Ax = h cannot be consistent for every 
possible b. 


An Overdetermined System 


The linear system 

*1 - 2*2 = *i 
*1- *2=^2 
*1 +- * 2=*3 
*1 +- 2*2 = *4 

*1 f 3*2 = *5 

is overdetermined, so it cannot be consistent for all possible values of £j, by by * 4 , and by Exact conditions under which the 
system is consistent can be obtained by solving the linear system by Gauss-Jordan elimination. We leave it for the reader to 
show that the augmented matrix is row equivalent to 

’l 0 2 b 2 - *i" 

0 1 b 2 — b \ 

0 0 *3 — 3 b 2 4 2 b \ 

0 0 *4 — 4 b 2 4 3 b \ 

0 0 *5 — 5 b 2 4 4 b i 


Thus, the system is consistent if and only if by by b^ and b$ satisfy the conditions 

2*i — 3b 2 4- £3 =0 


3b \ — 4b 2 
4b 1 -5*2 

or, on solving this homogeneous linear system, 

*1 = 5* —4s, b 2 =4r—3s, 

where r and s are arbitrary. 


4 £4 =0 

+ *5 = 0 

*3 = 2 * — s, *4 = *, * 5 =s 

♦ 


In Formula 3 of Theorem 5.5.2, the scalars cj, c 2 , ... , are the arbitrary parameters in the general solutions of both Ax = li 
and Ax = 0- Thus these two systems have the same number of parameters in their general solutions. Moreover, it follows from 
part (*) of Theorem 5.6.4 that the number of such parameters is nullity (A). This fact and the Dimension Theorem for Matrices 
(Theorem 5.6.3) yield the following theorem. 


THEOREM 5.6.7 


IfAx = h is a consistent linear system of m equations in n unknowns, and if A has rank r, then the general solution of the 
system contains n—r parameters. 


Number of Parameters in a General Solution 


If A is a 5 x 7 matrix with rank 4, and if Ax — b is a consistent linear system, then the general solution of the system contains 









7 — 4 = 3 parameters. 


♦ 

In earlier sections we obtained a wide range of conditions under which a homogeneous linear system Ax = 0 of n equations in n 
unknowns is guaranteed to have only the trivial solution. (See Theorem 4.3.4.) The following theorem obtains some 
corresponding results for systems of m equations in n unknowns, where m and n may differ. 


THEOREM 5.6.8 


If A is an mx n matrix, then the following are equivalent. 

( a ) Ax = 0 has only the trivial solution. 


(b) The column vectors of A are linearly independent. 


( c ) Ax = b has at most one solution (none or one) for every m x 1 matrix b. 


Proof It suffices to prove the two equivalences (a) ^ (£) and ( a ) (c), since it will then follow as a matter of logic that 

(*) ^ CO- 

If c 1 , c 2 , . • •, c H are the column vectors of A, then the linear system Ax = 0 can be written as 

-bx H o H = 0 (6) 

If ci, C 2 , ••c H are linearly independent vectors, then this equation is satisfied only by = X i = — — x n — Cu which means that 
Ax = 0 has only the trivial solution. Conversely, if Ax = 0 has only the trivial solution, then Equation 6 is satisfied only by 
*1 = *2 =■■■ = *„ = 0 > which means that cj, C 2 , • • • , c H are linearly independent. 

Assume that = 0 has only the trivial solution. Either yj x = lj is consistent or it is not. If it is not consistent, then 
there are no solutions of j 4 x = 1 ,, and we are done, if yj x — 1 , is consistent, let xo be any solution. From the discussion following 
Theorem 5.5.2 and the fact that Ax — 0 has only the trivial solution, we conclude that the general solution of Ax — b is 
XQ -|- 0 = XQ- Thus the only solution of j 4 x = b is xq. 

(c)=> Assume that j 4 x = b has at most one solution for every m x 1 matrix b. Then, in particular, Ax = b has at most one 
solution. Thus Ax = 0 has only the trivial solution. 


A linear system with more unknowns than equations is called an underdetermined linear system. If Ax = b is a consistent 
underdetermined linear system of m equations in n unknowns (so that m < «), then it follows from Theorem 5.6.7 that the 
general solution has at least one parameter (why?); hence a consistent underdetermined linear system must have infinitely many 
solutions. In particular, an underdetermined homogeneous linear system has infinitely many solutions, though this was already 
proved in Chapter 1 (Theorem 1.2.1). 


An Underdetermined System 





If A is a 5 x 7 matrix, then for every 7x1 matrix b, the linear system Ax = h is underdetermined. Thus Ax = lj must be 
consistent for some b, and for each such b the general solution must have 1 —r parameters, where r is the rank of A. 

Summary 

In Theorem 4.3.4 we listed eight results that are equivalent to the invertibility of a matrix A. We conclude this section by 
adding eight more results to that list to produce the following theorem, which relates all of the major topics we have studied 
thus far. 

THEOREM 5.6.9 

Equivalent Statements 

If A is an n matrix, and if T A : Pf 1 -► R n is multiplication by A, then the following are equivalent. 

(a) A is invertible. 

(b) Ax = 0 has only the trivial solution. 

(c) The reduced row-echelon form of A is / . 

(d) A is expressible as a product of elementary matrices. 

( e ) Ax = I is consistent for every n x 1 matrix b. 

(f) Ax = h has exactly one solution for every nx 1 matrix b. 

(g) det(A) 5* 0- 

(h) The range of Tj\ is pf 1 - 

(i) p A is one-to-one. 

(j) The column vectors of A are linearly independent. 

(k) The row vectors of A are linearly independent. 

(l) The column vectors of A span pf. 




(m) The row vectors of A span R n . 


(n) The column vectors of A form a basis for R n . 


(o) The row vectors of A form a basis for R n . 


(p) A has rank n. 


(q) A has nullity 0. 


Proof We already know from Theorem 4.3.4 that statements (a) through (i ) are equivalent. To complete the proof, we will 
show that (j) through (q) are equivalent to (b) by proving the sequence of implications 

Q>) - U) =* (*) => C l ) =* («) => (») =* ip) => ip) => iq) => {b). 

If Ax — 0 has only the trivial solution, then by Theorem 5.6.8, the column vectors of A are linearly independent. 

This follows from Theorem 5.4.5 and the fact that R n is an n-dimensional vector space. 

(The details are omitted.) 

If the n row vectors of A form a basis for R n , then the row space of A is n-dimensional and A has rank n. 

This follows from the Dimension Theorem (Theorem 5.6.3). 

If A has nullity 0, then the solution space of = 0 has dimension 0, which means that it contains only the zero 
vector. Hence = 0 has only the trivial solution. 


Exercise Set 5.6 



Click here for Just Ask! 


1 . 


Verify that rank(j4) = ranked T ). 


A = 


12 4 0 
-3152 
-2 3 9 2 


2. 


Find the rank and nullity of the matrix; then verify that the values obtained satisfy Formula 4 of the Dimension Theorem. 







(a) 1-1 3 

A= 5 -4 -4 

7-6 2 

(b) [2 0-1' 
A= 4 0 -2 

0 0 0 


(C) 

1 4 5 2' 

A = 

2 13 0 


-13 2 2 


1 4 5 6 9 

3-2 1 4-1 

-1 0 -1 -2 -1 

2 3 5 7 8 

(e) 1-3 2 

0 3 6 

A= 2 -3 -2 
3-6 0 

-2 9 2 


In each part of Exercise 2, use the results obtained to find the number of leading variables and the number of parameters in 

3. the solution of Ax = 0 without solving the system. 

In each part, use the information in the table to find the dimension of the row space, column space, and nullspace of A, and 

4. of the nullspace of a T - 



(a) 

(b) 

(c) 

(d) 

(e) 

(f) 

(g) 

Size of A 

3x3 

3x3 

3x3 

5x9 

9x5 4x4 

6x2 

Rank(A) 

3 

2 

1 

2 

2 

0 

2 


In each part, find the largest possible value for the rank of A and the smallest possible value for the nullity of 


(a) A is 4x4 

(b) A is 3 x 5 




5. 






(c) A is 5 x 3 


If A is an m x n matrix, what are the largest possible value for its rank and the smallest possible value for its nullity? 

6 . 


Hint See Exercise 5. 


In each part, use the information in the table to determine whether the linear system Ax = b is consistent. If so, state the 
7. number of parameters in its general solution. 



(a) 

(b) 

(c) 

(d) 

(e) 

(f) 

(g) 

Size of A 

3x3 

3x3 

3x3 

5x9 

5x9 

4x4 

6x2 

Rank(A) 

3 

2 

1 

2 

2 

0 

2 

Rank [A | b] 

3 

3 

1 

2 

3 

0 

2 


For each of the matrices in Exercise 7, find the nullity of A, and determine the number of parameters in the general solution 
8- of the homogeneous linear system Ax — 0- 


What conditions must be satisfied by hy b$> ar| d hr for the overdetermined linear system 

xi - 3x2 — 
xi - 2x2 =i>2 
xi +X2=&3 
xi -4x2 =^4 
xi 4- 5x2 =£5 

to be consistent? 


Let 


10 . 


a n a 12 
a 2\ a 22 


a 13 
a 23 


Show that A has rank 2 if and only if one or more of the determinants 


a 11 a \2 


flu <313 


a 12 «13 

a 21 a 22 


a 21 <*23 


<*22 ®22 


are nonzero. 


Suppose that A is a 3 x 3 matrix whose nullspace is a line through the origin in 3-space. Can the row or column space of A 
11* also be a line through the origin? Explain. 


12. 


Discuss how the rank of A varies with 





(a) 


A = 


1 

1 


t 


1 t 
t 1 
1 1 


(b) t 3-1 

A = 3 6-2 

-1-3 £ 


Are there values of r and s for which 

0 0 

r- 2 2 

s— 1 r+ 2 

0 3 

has rank 1 or 2? If so, find those values. 

of points (x,y,z) in p} for which the matrix 
z~ 

has rank 1 is the curve with parametric equations x =t, y — P‘, z -p- 

Prove: If fc ^ Q, then A and fcA have the same rank. 

15. 


14. 


Use the result in Exercise 10 to show that the set 

* y 

1 X 


o 

o 

o 


Discussion 

lUt 

(a) Give an example of a 3 x 3 matrix whose column space is a plane through the origin in 
3-space. 

(b) What kind of geometric object is the nullspace of your matrix? 

(c) What kind of geometric object is the row space of your matrix? 

(d) In general, if the column space of a 3 x 3 matrix is a plane through the origin in 3-space, 
what can you say about the geometric properties of the nullspace and row space? Explain 
your reasoning. 


Indicate whether each statement is always true or sometimes false. Justify your answer by givinj 
17. a logical argument or a counterexample. 


(a) If A is not square, then the row vectors of A must be linearly dependent. 












(b) If A is square, then either the row vectors or the column vectors of A must be linearly 
independent. 

(c) If the row vectors and the column vectors of A are linearly independent, then A must be 
square. 

(d) Adding one additional column to a matrix A increases its rank by one. 


18. 

(a) If A is a 3 x 5 matrix, then the number of leading l's in the reduced row-echelon form of 
A is at most_. Why? 


(b) If A is a 3 x 5 matrix, then the number of parameters in the general solution of Ax = 0 is 
at most_. Why? 


(c) If A is a 5 x 3 matrix, then the number of leading l's in the reduced row-echelon form of 
A is at most_. Why? 


(d) If A is a 5 x 3 matrix, then the number of parameters in the general solution of Ax = 0 is 
at most_. Why? 


19. 

(a) If A is a 3 x 5 matrix, then the rank of A is at most_. Why? 


(b) If A is a 3 x 5 matrix, then the nullity of A is at most_. Why? 

(c) If A is a 3 x 5 matrix, then the rank of is at most_. Why? 


(d) If A is a 3 x 5 matrix, then the nullity of A T is at most_. Why? 
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Abbreviations 


C cyan K black M magenta Y yellow 
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Chapter 5 


Supplementary Exercises 


In each part, the solution space is a subspace of and so must be a line through the origin, a plane through the origin, all 
of g?, or the origin only. For each system, determine which is the case. If the subspace is a plane, find an equation for it, 
and if it is a line, find parametric equations. 

(a) Ox 4 - Oy 4 Oz = 0 

(b) 2 x — 3y 4- z — 0 
6 x - 9 y 4 3z = 0 

— Ax 4 6 y — 2z = 0 

(c) x - 2 y 4 Iz = 0 
- 4x 4 Sy 4 5z = 0 

2x — Ay 4 3z = 0 

(d) x ■+■ Ay q 8z = 0 
2x + 5y + 6z = 0 
3x 4 - y - 4z = 0 


For what values of s is the solution space of 

xi 4 X 2 4- SX 3 = 0 
xi sx2 4x3 = 0 
sxi 4x2 4x3 = 0 

the origin only, a line through the origin, a plane through the origin, or all of 


3. 

(a) Express ( 4a, a — b,a-V 2b) as a linear combination of (4, 1, 1) and (0,-1, 2). 


(b) Express (3 a 4 b 4 3c, - a 4 4b - c, 2a 4 b 4 - 2c) as a linear combination of (3, -1, 2) and (1, 4, 1). 


(c) Express (2a — b 4 Ac, 3a — c, 4 b 4 c) as a linear combination of three nonzero vectors. 


4. 


Let W be the space spanned by f = sin* and g = cos x. 



(a) Show that for any value of f ^ = sin(^ 4- 8) and ^ = cos (x + 9) are vectors in W. 

(b) Show that f 1 and ^ form a basis for W. 


5 . 

(a) Express v = ( 1 , 1 ) as a linear combination of = (1, — 1 ), V 2 = (3, 0), and V 3 = (2, 1) in two different ways. 

(b) Explain why this does not violate Theorem 5.4.1. 

Let A be an n x n matrix, and let vi, V 2 , ... v H be linearly independent vectors in R n expressed as n x 1 matrices. What 
6 * must be true about A for Av\, Av 2 * •••, to be linearly independent? 

Must a basis for p contain a polynomial of degree k for each k = Cf L 2 , ..., nl Justify your answer. 

7. 

For purposes of this problem, let us define a “checkerboard matrix” to be a square matrix A — [a,-,-] such that 

8 - f, ■ 

1 if i 4 j is even 

j 0 if j 4 j is odd 

Find the rank and nullity of the following checkerboard matrices: 

(a) the 3 x 3 checkerboard matrix 

(b) the 4 x 4 checkerboard matrix 

(c) the n xn checkerboard matrix 


For purposes of this exercise, let us define an “ X-matrix” to be a square matrix with an odd number of rows and column 
that has 0's everywhere except on the two diagonals, where it has l's. Find the rank and nullity of the following 
X-matrices: 

(a) [1 0 r 

0 1 0 
1 0 1 

(b) Tl 0 0 0 1 

0 10 10 
0 0 10 0 
0 10 10 
1 0 0 0 1 



(c) the X-matrix of size (2 m + 1) x (2 m -4 1) 


In each part, show that the set of polynomials is a subspace of p and find a basis for it. 

10 . 

(a) all polynomials in p such that p( — x) — p(x) 

(b) all polynomials in p such that t?(0) = 0 


11. (For Readers Who Have Studied Calculus) Show that the set of all polynomials in p that have a horizontal tangent 
at x — 0 is a subspace of p . Find a basis for this subspace. 


12 . 

(a) Find a basis for the vector space of all 3x3 symmetric matrices. 


(b) Find a basis for the vector space of all 3x3 skew-symmetric matrices. 


In advanced linear algebra, one proves the following determinant criterion for rank: The rank of a matrix A is r if and 
13. only if A has some rxr submatrix with a nonzero determinant, and all square submatrices of larger size have 

determinant zero. (A submatrix of A is any matrix obtained by deleting rows or columns of A. The matrix A itself is also 
considered to be a submatrix of A.) In each part, use this criterion to find the rank of the matrix. 


(a) 


1 2 
2 4 


(b) 


1 2 3 

2 4 6 


(c) 


1 

2 

3 


0 1 
-1 3 
-1 4 


(d) 


1 -12 0 
3 10 0 

-12 4 0 


14. 


Use the result in Exercise 13 to find the possible ranks for matrices of the form 





«16 
32 6 


33 6 
346 
3 56 

Prove: If 5 is a basis for a vector space V, then for any vectors u and v in V and any scalar k, the following relationships 
15. hold: 

(a) (u 4 v) rj = (u) s | (v) s 

(b) (in) (S . = t(u) (S 
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0 0 0 0 0 

0 0 0 0 0 

0 0 0 0 0 

0 0 0 0 0 

351 3^2 353 £354 £355 











Chapter 5 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Section 5.2 

Tl. 

(a) Some technology utilities do not have direct commands for finding linear combinations of vectors in R }1 . However, 
you can use matrix multiplication to calculate a linear combination by creating a matrix A with the vectors as columns 
and a column vector x with the coefficients as entries. Use this method to compute the vector 

v = 6(8, -2,1, —4) + 17( — 3, 9, 11, 6)-9(0, -1,2,4) 

Check your work by hand. 

(b) Use your technology utility to determine whether the vector (9, 1, 0) is a linear combination of the vectors (1, 2, 3), (1, 
4, 6), and (2, -3, -5). 


Section 5.3 

Use your technology utility to perform the Wronskian test of linear independence on the sets in Exercise 20. 

Tl. 

Section 5.4 

Tl. (Linear Independence) Devise three different procedures for using your technology utility to determine whether a set of n 
vectors in R n is linearly independent, and use all of your procedures to determine whether the vectors 

vi = (4, -5,2,6), v? = (2, — 2, 1, 3), v? = (6, - 3, 3, 9), v d = (4, - 1, 5, 6) 

are linearly independent. 

T2. (Dimension) Devise three different procedures for using your technology utility to determine the dimension of the subspace 
spanned by a set of vectors in R n , and use all of your procedures to determine the dimension of the subspace of r} spanned 
by the vectors 


Section 5.5 


VI = (2, 2, - 1, 0, 1), v 2 = ( - 1, - 1, 2, - 3, 1), 
v 3 =(U, -2,0, -1), v 4 = (0,0, 1,1,1) 



Tl. (Basis for Row Space) Some technology utilities provide a command for finding a basis for the row space of a matrix. If 
your utility has this capability, read the documentation and then use your utility to find a basis for the row space of the matrix 
in Example 6. 


T2. (Basis for Column Space) Some technology utilities provide a command for finding a basis for the column space of a 

matrix. If your utility has this capability, read the documentation and then use your utility to find a basis for the column space 
of the matrix in Example 6. 


T3. (Nullspace) Some technology utilities provide a command for finding a basis for the nullspace of a matrix. If your utility has 
this capability, read the documentation and then check your understanding of the procedure by finding a basis for the 
nullspace of the matrix A in Example 4. Use this result to find the general solution of the homogeneous system Ax = 0- 


Section 5.6 

Tl. (Rank and Nullity) Read your documentation on finding the rank of a matrix, and then use your utility to find the rank of 
the matrix A in Example 1. Find the nullity of the matrix using Theorem 5.6.3 and the rank. 

There is a result, called Sylvester's inequality, which states that if A and B are nxn matrices with rank rj± and rg, 

T2. respectively, then the rank r^g of AB satisfies the inequality f rg — n < rjg < icmirji rg ), where (r^ rg) denotes the 
smaller of r j and rg or their common value if the two ranks are the same. Use your technology utility to confirm this result 
for some matrices of your choice. 
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6 

CHAPTER 


Inner Product Spaces 


In Section 3.3 we defined the Euclidean inner product on the spaces R 2 and j? 3 . Then, in Section 4.1, 
we extended that concept to R H and used it to define notions of length, distance, and angle in R n . In this section we shall 
extend the concept of an inner product still further by extracting the most important properties of the Euclidean inner product on 
R n and turning them into axioms that are applicable in general vector spaces. Thus, when these axioms are satisfied, they will 
produce generalized inner products that automatically have the most important properties of Euclidean inner products. It will 
then be reasonable to use these generalized inner products to define notions of length, distance, and angle in general vector 
spaces. 
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6.1 

INNER PRODUCTS 


In this section we shall use the most important properties of the Euclidean inner 
product as axioms for defining the general concept of an inner product. We will 
then show how an inner product can be used to define notions of length and 
distance in vector spaces other than R n . 


General Inner Products 

In Section 4.1 we denoted the Euclidean inner product of two vectors in R n by the notation u . v . It will be convenient in this 
section to introduce the alternative notation {u, v) for the general inner product. With this new notation, the fundamental properties 
of the Euclidean inner product that were listed in Theorem 4.1.2 are precisely the axioms in the following definition. 


DEFINITION 


An inner product on a real vector space V is a function that associates a real number (u, v) with each pair of vectors u and v in 
V in such a way that the following axioms are satisfied for all vectors u , v, and z in V and all scalars k. 

1. (u, v} = {v, u} [Symmetry axiom] 

2. ( u + v, z) = {u, z) 4 (v, z) [Additivity axiom] 

3. (An, v) = £{u, v) [Homogeneity axiom] 

4. {v, v) > 0 [Positivity axiom] 

and (v, v) = 0 
if and only if v = 0 

A real vector space with an inner product is called a real inner product space. 


In Chapter 10 we shall study inner products over complex vector spaces. However, until that time we shall use the term 
inner product space to mean “real inner product space.” 


Because the inner product axioms are based on properties of the Euclidean inner product, the Euclidean inner product satisfies 
these axioms; this is the content of the following example. 


Euclidean Inner Product on R n 

If u = (ui, U 2 , U„) and v = ( Vl , V 2 ,v H ) are vectors in pj 1 , then the formula 

{u, v } = u - v = ui Vi f U 2 V 2 4 — 4 u n v n 

defines (u, v) to be the Euclidean inner product on R n . The four inner product axioms hold by Theorem 4.1.2. 










The Euclidean inner product is the most important inner product on R n . However, there are various applications in which it is 
desirable to modify the Euclidean inner product by weighting its terms differently. More precisely, if 

w\,W 2 ,...,w n 

are positive real numbers, which we shall call weights, and if u = (zq, U 2 , and v = (vi, V 2 , v H ) are vectors in R n , then it 

can be shown (Exercise 26) that the formula 

(u, v) = wiui VI +W2U2V2 + -w„u n v n ^ 

defines an inner product on R n : it is called the weighted Euclidean inner product with weights w\, nq ■■■, w n . 


To illustrate one way in which a weighted Euclidean inner product can arise, suppose that some physical experiment can produce 
any of n possible numerical values 


xi,X2,-.,x„ 


and that a series of m repetitions of the experiment yields these values with various frequencies; that is, jq occurs f j times, jq 
occurs j 2 times, and so forth. Since there are a total of m repetitions of the experiment, 

/ 1 4 / 2 H-1 ' f n = m 


Thus the arithmetic average, or mean, of the observed numerical values (denoted by x) is 


= f\x\ I /2*2d “■ I 

/1 + J2 b-+/ H 


= ^(/l^l-l f2X2+- + fnXyi) 


( 2 ) 


If we let 

X= x„) 

W 1 = W2 = — = = 1 / m 

then 2 can be expressed as the weighted inner product 

* = <f, x) = *1/1*1 H ^2/2^2 1-1 


Remark It will always be assumed that R n has the Euclidean inner product unless some other inner product is explicitly specified. 
As defined in Section 4.1, we refer to R n with the Euclidean inner product as Euclidean n-space. 


Weighted Euclidean Inner Product 


Let u = (u\, U'i) and v = (v 1 , V 2 ) be vectors in r\ Verify that the weighted Euclidean inner product 

(u, v) = 3u\ vi 4 2^2 ^2 


satisfies the four inner product axioms. 


Solution 


Note first that if u and v are interchanged in this equation, the right side remains the same. Therefore, 

(u, v) = (v, u) 


If 1 = (z\, Z 2 )’ then 


which establishes the second axiom. 


(u -I- v, z), = 3(ui + v\)z\ + 2 {u 2 4- v 2 )z 2 
= (3iqzi + 2u 2 z 2 ) 4 (3vizi f 2v 2 Z2) 
= in, zj 4 (v, zj 





Next, 


{£u, v) = 3(&wi)vi 4- 2(&£*2) v 2 =k(3u \ vi + 2^2 ^ 2 ) = £{u, v) 
which establishes the third axiom. 


Finally, 


(v, v) = 3vi vi 4 2v2 V 2 = 3v 2 4- 2v| 

Obviously, ,; Vi v j _ 2 V \ + 2v| > 0- Further, v j = 4. 2vj = 0 if and onl y if vi = v 2 = 0—that is, if and only if 

v = (vi, V 2 ) = 0- Thus the fourth axiom is satisfied. 


♦ 


Length and Distance in Inner Product Spaces 

Before discussing more examples of inner products, we shall pause to explain how inner products are used to introduce notions of 
length and distance in inner product spaces. Recall that in Euclidean n-space the Euclidean length of a vector u = (it\, 112 , u n ) 
can be expressed in terms of the Euclidean inner product as 

Hull = (u-u) 1/2 

and the Euclidean distance between two arbitrary points u = (u\, 112 , u n ) and v = (vj, V 2 ,v H ) can be expressed as 

d (u, v) = ||u - v|| = [ (u - v) - (u - v) ] 1 n 

[see Formulas 1 and 2 of Section 4.1]. Motivated by these formulas, we make the following definition. 


DEFINITION 


If V is an inner product space, then the norm (or length ) of a vector u in Vis denoted by ||u|| and is defined by 


Hull = (u, u) 1/2 

The distance between two points (vectors) u and v is denoted by d (u, v) an( 3 is defined by 


> 

1 

_s_ 

II 

> 

^3 


If a vector has norm 1, then we say that it is a unit vector. 


EXAMPLE 3 Norm and Distance in R n 


If u = {u\,U 2 , and v = (vi, V 2 ,v H ) are vectors in P. n with the Euclidean inner product, then 

Hull ={u,u) 1/2 = (u-u) 1/2 = u| + -. + u 2 
and 

i 1 /2 

d (u, v) = ||u - v|| = ju - v, u - v j = [ (u - v) - (u - v) ] 1 /2 
= {{u\ -vi) 2 + (u 2 — v 2 ) 2 H- ~V (u H -v M ) 2 

Observe that these are simply the standard formulas for the Euclidean norm and distance discussed in Section 4.1 [see Formulas 1 
and 2 in that section]. 












EXAM Using a Weighted Euclidean Inner Product 


It is important to keep in mind that norm and distance depend on the inner product being used. If the inner product is changed, then 
the norms and distances between vectors also change. For example, for the vectors u = (1, 0) and v =(0, l)in)? 2 with the 
Euclidean inner product, we have 

INI = i/i 2 + o 2 = l 

and 

rf(u,v) = ||u-v|| = ||(l, -l)|| = i/l 2 + (-l) 2 = /2 
However, if we change to the weighted Euclidean inner product of Example 2, 

{u, v) = 3u\ vi A 2u2 V 2 


then we obtain 

|H| = {\i, u) 1/2 = [3(1)(1) + 2(0)(0)] 1/2 = fe 
and 

d(n,v) = ||n-v|| = j(l, - 1), (1, - 1)J 
= [3(l)(l) + 2(-l)(-l)] 1/2 = /5 

♦ 


Unit Circles and Spheres in Inner Product Spaces 

If V is an inner product space, then the set of points in V that satisfy 

INII = l 

is called the unit sphere or sometimes the unit circle in V. In p} and these are the points that lie 1 unit away from the origin. 


Unusual Unit Circles in R? 


(a) Sketch the unit circle in an xy-coordinate system in R 2 using the Euclidean inner product ( u , v ) = u i vi +U2 V 2 - 

(b) Sketch the unit circle in an xy-coordinate system in R 2 using the weighted Euclidean inner product 

(u, v) = ^ui vi H ^u 2 v 2 - 


Solution (a) 

If u = (x, y), then ||u|| = {u, u} 1/2 = yjjp" +y 2 so the equation of the unit circle is + y 2 = 1» or, on squaring both sides, 

x 2 H y 2 = 1 

As expected, the graph of this equation is a circle of radius 1 centered at the origin (Figure 6.1.1a). 









it*} The uniuirclc wiih 
iIk 1 Euclid mil norm 

Hufl = V.r + r 



(/?) The unii circle with 

the norm_ 

null = V jjr + j y 2 


Figure 6.1.1 


Solution (b) 

If u = (x, 7 ), then ||u|| = (u, u} 1/2 = \j-^x^-\ so the equation of the unit circle is ^4 iy 2 = 1, or, on squaring both 


sides, 


, 2 , ^ 


Q 4 

The graph of this equation is the ellipse shown in Figure 6.1.1b. 


= 1 


It would be reasonable for you to feel uncomfortable with the results in the last example, because although our definitions of 
length and distance reduce to the standard definitions when applied to R l with the Euclidean inner product, it does require a stretch 
of the imagination to think of the unit “circle” as having an elliptical shape. However, even though nonstandard inner products 
distort familiar spaces and lead to strange values for lengths and distances, many of the basic theorems of Euclidean geometry 
continue to apply in these unusual spaces. For example, it is a basic fact in Euclidean geometry that the sum of the lengths of two 
sides of a triangle is at least as large as the length of the third side (Figure 6.1.2a). We shall see later that this familiar result holds 
in all inner product spaces, regardless of how unusual the inner product might be. As another example, recall the theorem from 
Euclidean geometry that states that the sum of the squares of the diagonals of a parallelogram is equal to the sum of the squares of 
the four sides (Figure 6.1.2b). This result also holds in all inner product spaces, regardless of the inner product (Exercise 20). 



Figure 6.1.2 


Inner Products Generated by Matrices 










The Euclidean inner product and the weighted Euclidean inner products are special cases of a general class of inner products on R n 
, which we shall now describe. Let 



‘«l‘ 


■vf 

u = 

u 2 

g 

II 

V 2 


U Yi 




be vectors in R }1 (expressed as n x 1 matrices), and let A be an invertible n x.n matrix. It can be shown (Exercise 30) that if u . v is 
the Euclidean inner product on R n , then the formula 


(u, v} = An-A y 

defines an inner product; it is called the inner product on R s generated by A. 


(3) 


Recalling that the Euclidean inner product u. v can be written as the matrix product v ^u [see 7 in Section 4.1], it follows that 3 
can be written in the alternative form 


or, equivalently, 


(u, v) = (Ay) * An 


(u, v) = A* An 


(4) 


Inner Product Generated by the Identity Matrix 


The inner product on R n generated by the n x.n identity matrix is the Euclidean inner product, since substituting A = I in 3 yields 

(u, v) = /u - / V = 11 ■ V 

The weighted Euclidean inner product f u , v ) = 3u\v\ + 2h 2 V2 discussed in Example 2 is the inner product on r} generated by 

j/3 0 


A = 


0 


because substituting this matrix in 4 yields 


= [vi V 2 ] 


= [vi V 2 ] 

'3 O' 

.0 2 _ 

= vi - 1 - 2 u 2 v 2 


_1 

-1 

O 

_1 

■«r 

1_ 

1- 

O 

"kT| 

1_ 

u 2 


“ 1 
u 2 


In general, the weighted Euclidean inner product 

(11, v) = vi? 1 u 1 VI 4- w 2 U2 V2 h ™ + w„u H v H 

is the inner product on R n generated by 

\fw[ 0 0 — 0 


A = 


0 ~2 0 — 0 
0 0 0 — 


(5) 


(verify). 






















In the following examples we shall describe some inner products on vector spaces other than R n . 


An Inner Product on M 22 


If 



u 2 

U 4 


and 



v 2 

v 4 


are any two 2 x 2 matrices, then the following formula defines an inner product on Mji (verify): 

{ U, V) = tr(U^ F) = tr(V^U) = u\ vj 4 - uj V 2 4- U 2 V 3 4 - 114 v 4 
(Refer to Section 1.3 for the definition of the trace.) For example, if 


U = 


1 2 
3 4 


and 



0 

2 


then 

{U, r> = l(-l) + 2(0) +3(3)+ 4(2) = 16 
The norm of a matrix U relative to this inner product is 

\\U\\ ={U, U) m = / u 2 4a 2 + a 2 Vu 2 

and the unit sphere in this space consists of all 2 x 2 matrices U whose entries satisfy the equation || U\\ = 1, which on squaring 
yields 

2 , 2 . 2 . 2 1 

^1 u 2 u 3 u 4 — ^ 


♦ 


EXAMPLE 8 An Inner Product on p 2 

If 

2 2 
p = flo4flixH ct 2 X and q = &o 4- &l x 4- &2 x 

are any two vectors in p 2 , then the following formula defines an inner product on p 2 (verify): 

<1>, q)=«o^O+«l^l ffl2^2 

The norm of the polynomial p relative to this inner product is 

IIPII = (P> P ) 1/2 = |/fln T a 2 + a 2 

and the unit sphere in this space consists of all polynomials p in p 2 whose coefficients satisfy the equation ||/>|| = 1 , which on 
squaring yields 

2 1 2 1 2 -1 

i3q + 1 3^ -f ^2 — ^ 


Calculus Required 


An Inner Product on C [a, b\ 



















Let f = f (x) and g = g{x) be two functions in C[a,b] and define 


( f , S }=/J 




( 6 ) 


This is well-defined since the functions in C[a, &] are continuous. We shall show that this formula defines an inner product on 
C[a, &] by verifying the four inner product axioms for functions f = /(*), g = g(x), and 3 = s(x) in C[a, b\. 


1 . 


{f. g)=/ /(*)gOO^=/ g(*)/(*y* = (g,f} 

Ja Ja 

which proves that Axiom 1 holds. 


2 . 


(f + g,s}= / 

Ja 


C/OO + g(x))s(x)dx 


fb rb 

= / f{x)s{x)dx + / g{x)s{x)dx 
Ja Ja 

= { f ,s} + (g,s} 

which proves that Axiom 2 holds. 

(*f. g}=/ kf{x)g{x)dx=kj f{x)g{x)dx=k[ f, g} 

Ja Ja 

which proves that Axiom 3 holds. 

If f = j (*) is any function in C[a,b], then f 2 {x) >0 for all x in [a,b] \ therefore, 


(f, f} = j j' 2 {x)dx > 0 

Further, because f 2 {x) > 0 and f = /(*) is continuous on [a, b], it follows that fnf 2 {x)dx = 0 if and 

fb 

only if f(x) = 0 for all x in \ a , b\. Therefore, we have (f, f} = / f 2 (x)dx = 0 if and only if f = 0- This 

Ja 


proves that Axiom 4 holds. 


Calculus Required 


Norm of a Vector in C[a, £>] 


If CTa, &1 has the inner product defined in the preceding example, then the norm of a function f = f (x) relative to this inner 
product is 


||f|| = (f,f } 1/2 


f z (x)dx 


(7) 


and the unit sphere in this space consists of all functions fm.C[a,b] that satisfy the equation ||f || = 1, which on squaring yields 







Calculus Required 


Remark Since polynomials are continuous functions on ( _ M M they are continuous on any closed interval [a,b]. Thus, 
for all such intervals the vector space p n is a subspace of C[a, & ], and Formula 6 defines an inner product on p n . 


Calculus Required 


Recall from calculus that the arc length of a curve y = f (x) over an interval \a,b~\ is given by the formula 



( 8 ) 


Do not confuse this concept of arc length with ||f ||, which is the length (norm) of/when/is viewed as a vector in C\a, 61 • 
Formulas 7 and 8 are quite different. 

The following theorem lists some basic algebraic properties of inner products. 

THEOREM 6.1.1 

Properties of Inner Products 

If u, v, and w are vectors in a real inner product space, and k is any scalar, then 

(a) {0, v) = {v,0) = 0 

(b) (u, v -I- w) = {u, v) + (u, w) 

(c) (u, kv)=k{n, v) 

(d) {« - v, w) = {u, w) - (v, w) 

(e) («, v - w) = (u, v) - {u, w) 

Proof We shall prove part (£) and leave the proofs of the remaining parts as exercises. 


(u, v 4- wj, = (v f w, uj [By sviranetiy ] 


= (v, uj 4- (w, uj, [By additivity] 
= (\\ ? v) + In, wj, [By symmetry] 






The following example illustrates how Theorem 6.1.1 and the defining properties of inner products can be used to perform 
algebraic computations with inner products. As you read through the example, you will find it instructive to justify the steps. 


Calculating with Inner Products 


{u - 2v, 3u 4 4v) = (u, 3u 4 4v) - {2v, 3u + 4v) 

= (u, 3u> 4 {u, 4v) - {2v, 3u) - (2v, 4v> 

= 3{u, u) -4 4{u, v) — 6{v, u} — 8{v, v) 

= 3 ||u|| 2 + 4{u, v) - 6{u, v) - 8||v|| 2 
= 311it11 2 — 2{u, v) — S||v|| 2 

♦ 

Since Theorem 6.1.1 is a general result, it is guaranteed to hold for all real inner product spaces. This is the real power of the 

axiomatic development of vector spaces and inner products—a single theorem proves a multitude of results at once. For example, 

we are guaranteed without any further proof that the five properties given in Theorem 6.1.1 are true for the inner product on R n 
generated by any matrix A [Formula 3], For example, let us check part ( b ) of Theorem 6.1.1 for this inner product: 

(u, v 4- w} = (v 4 w) ^A ^ An 

7" 1 'T 7" 1 

= (v 4w )A Au [Property of transpose] 

= (v^ A^ Au) I (w 7 " A 7 " An) [Property of matrix multiplication] 

= (u, vj 4 fu, wj 

The reader will find it instructive to check the remaining parts of Theorem 6.1.1 for this inner product. 


Exercise Set 6.1 



Click here for Just Ask! 


Let (u, v) be the Euclidean inner product on and let u = (3, — 2), v = (4, 5), w = ( — 1, 6), and k= -4- Verify that 

(a) {u,v) = {v, u) 

(b) (u 4 v, w) = (u, w) 4 (v, w) 

(c) {u, v 4 w) = (u, v) 4 {u, w) 

(d) {jfcu, v) = £{ 11 , v) = (u, £v) 


(e) (0, v) = (v, 0) = 0 





Repeat Exercise 1 for the weighted Euclidean inner product (u, v) = 4u\ vj 4- 5u 2 v 2 


Compute { 11 , v) using the inner product in Example 7. 

3. 


(a) 

'3 

-2' 


'-1 3' 

u = 

_4 

8_ 

, v = 

1 1 


(b) 

11 = 

1 2 

, v = 

'4 

6' 


-3 5 


0 

8 


Compute {p, q) using the inner product in Example 8. 

4. 

(a) p = — 2 + x 4 3x 2 , q = 4 — 7x 2 

(b) p = _ 5 + 2x 4 x 2 , q = 3 + 2x - 4x 2 


5. 

(a) Use Formula 3 to show that v) = 9uj vi 4- V2 the i nner product on p} generated by 


A = 


3 0 
0 2 


(b) Use the inner product in part (a) to compute { 11 , v)if u = ( — 3, 2) and v = (1,7). 


6 . 

(a) Use Formula 3 to show that v) = 5u \ vi — u \ v 2 — u 2 vi 4 ltU? V 2 i s the inner product on p} generated by 



(b) Use the inner product in part (a) to compute {11, v)if u= (0, — 3) and v = (6, 2)- 


Let u = (uj, u'i) and v = (v 1 , V 2 )- In eac h part, the given expression is an inner product on p}. Find a matrix that generates it. 


(a) {u, v) = 3u \ vi 4- 5u 2 V 2 


(b) {u, v) = 4u\ vi + &u 2 v 2 



Let u = (uj, u 2 ) an( J v = (vi, V 2 )- Show that the following are inner products on p} by verifying that the inner product axioms 
hold. 


(a) {u, v) = 2u \ vi 4- 5u 2 v 2 


(b) {11, v) = 4uivi + u 2 v\ f V 2 -I 4^2 v 2 

Let n= (u\,u 2 ,u 2 ) and v = (y\, v 2 , v 2 )- Determine which of the following are inner products on p}. For those that are not, list 
the axioms that do not hold. 


(a) {u, v) = u\ vi 4 W 3 V 3 


(b) , i 22 , 22,22 

K J {u, v) = Uj Vj 4 u 2 v 2 I “3 v 3 


(c) {u, v) = 2u \vi 4 u 2 v 2 -V Au 2 v 2 


(d) {u, v) = u\ vi — u 2 V 2 4- u 2 V 3 


In each part, use the given inner product on p} to find ||w||, where w = ( — 1 3 ). 

10 . " ' ' 


(a) the Euclidean inner product 


(b) the weighted Euclidean inner product ( u , v ) = 3u\ vi + 2u 2 v 2 , where u = ( Uli u 2 ) and v = (vi, V 2 ) 


(c) the inner product generated by the matrix 



Use the inner products in Exercise 10 to find d(ii, v) for u = ( — L 2) and v = (2, 5)- 

Let P-, have the inner product in Example 8 . In each part, find ||i>|| 

12 . 

(a) p = — 2 4- 3x 4 2x 3 


(b) p = 4 



13 . 


Let M-n have the inner product in Example 7. In each part, find ||j4||. 


(a) 

(b) 


A = 


A = 


-2 5 
3 6 


0 0 
0 0 


Let p 2 have the inner product in Example 8. Find d(\i, q)- 
p = 3 —x + x 2 , q = 2-|-5x 2 

Let M '12 have the inner product in Example 7. Find d(A, B')- 


(a) 


A = 


2 6 
9 4 


5 = 


-4 7 
1 6 


™A = 


2 4 
1 0 


B = 


-5 1 
6 2 


Suppose that u, v, and w are vectors such that 
16 ‘ (u, v) = 2, (v, w) = 3, (u, w) = 5, ||u|| = l, 11 v 11 = 2, 

Evaluate the given expression. 


(a) (u fv, v + w) 

(b) {2v — w, 3u + 2w} 

(c) (u - v - 2w, 4u 4- v) 

(d) H u + V || 

(e) || 2w — v|| 

(f) ||u-2v + 4w|| 


INI =7 


17 . 


(For Readers Who Have Studied Calculus) 

Let the vector space p 2 have the inner product 




p(x)q(x)dx 


(a) Find ||i>|| for p = 1, l> = x, p = x 1 . 

(b) Find q) if p = 1 and q = x. 


Sketch the unit circle in using the given inner product. 

18. 


^ {U,V) = ^1V1 + ^ 2 V 2 

(b) {u, v) = 2ui vi + u 2 v 2 


Find a weighted Euclidean inner product on R 2 for which the unit circle is the ellipse shown in the accompanying figure. 



Figure Ex-19 


20 . 


Show that the following identity holds for vectors in any inner product space. 
I|U + V || 2 + ||u — v|| 2 = 2||u|| 2 + 2|)vj] 2 


21 . 


Show that the following identity holds for vectors in any inner product space. 
(U, V} = ^ ||u 4- v|| 2 - ^ ||u - v|| 2 


22. Let ^ = 


u \ 

u 3 


u 7 

U4 


and V = 


vi 

v 3 


v 2 

v 4 


. Show that {U,V) = U\V\ I- u 2 v 3 + v 2 4 y 4 v 4 is not an inner P r °duct on m 22 - 


Let p = p(x) and q = q(x) be polynomials in p 2 - Show that 
(P. q)=M0)?C0)+^(I) l j(I) + Ml)-?Cl) 
is an inner product on p 2 . Is this an inner product on p 2 } . Explain. 

Prove: If {u, v) is the Euclidean inner product on R } \ and if A is an n x.n matrix, then 

{u, Ay) = {A T u, v) 

Hint Use the fact that (u, v) = u - v = v ^ u. 


24. 










Verify the result in Exercise 24 for the Euclidean inner product on p} and 


' - f 


3' 


'i -2 r 

2 

, v = 

0 

, A = 

3 4 0 

4 


-2 


5-12 


Let u = (u\, U 2 , u H ) an ^ v = (vi, V 2 , v H )- Show that 

26. 

(u, v) = W1 U 1 VI + U2 V2 H-h W„W„V„ 

is an inner product on i? M if wj, W 2 , are positive real numbers. 

27. (For Readers Who Have Studied Calculus) 

Use the inner product 

(lb q}= j J p(x)q(x)dx 

to compute {p, q), for the vectors p = p(x) and q = q(x) in Py 


(a) p = 1 — x f x 2 I 5x 3 , q = x — 3x 2 

(b) v = x -5x 3 , q = 2 I 8x 2 


28. (For Readers Who Have Studied Calculus) 

In each part, use the inner product 

. s}= J 

to compute {f, g), for the vectors { = f (x) and g = g(x) in C[ 0, 1 ]. 


(a) f = cos 2xx, g = sin 2xx 

(b) f = x, g = e x 

(c) f = tan ±x, g = 1 

Show that the inner product in Example 7 can be written as { U, V) = tr(J7 V). 

29. 

Prove that Formula 3 defines an inner product on R n . 

30. 

Hint Use the alternative version of Formula 3 given by 4. 




Show that matrix 5 generates the weighted Euclidean inner product (u, v) = wj m vi 4- ^2 V 2 4-h w n u n v M - 


31 . 


Discussion 

Discovery 


The following is a proof of part (c) of Theorem 6.1.1. Fill in each blank line with the name of an 
32. inner product axiom that justifies the step. 

Hypothesis: Let u and v be vectors in a real inner product space. 

Conclusion: {u, &v) = £{u, v). 

Proof: 

1. {u, kv) = {kv, u)_ 

2. = jfc{v,ll}_ 

3. = jfc{u, v]- 


Prove parts (a), ( d ), and (e) of Theorem 6.1.1, justifying each step with the name of a vector space 
33. axiom or by referring to previously established results. 


Create a weighted Euclidean inner product v} = au\v\ + bu 2 V 2 ° n P? f° r which the unit circle 
34* in an xy-coordinate system is the ellipse shown in the accompanying figure. 



Figure Ex-34 

Generalize the result of Problem 34 for an ellipse with semimajor axis a and semiminor axis b, with 
35. a and b positive. 


Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 










6.2 

ANGLE AND 
ORTHOGONALITY IN 
INNER PRODUCT 
SPACES 


In this section we shall define the notion of an angle between two vectors in 
an inner product space, and we shall use this concept to obtain some basic 
relations between vectors in an inner product, including a fundamental 
geometric relationship between the nullspace and column space of a matrix. 


Cauchy-Schwarz Inequality 

Recall from Formula 1 of Section 3.3 that if u and v are nonzero vectors in pf or p} and 0 is the angle between them, then 

u-v = ||u||||y||cos0 


or, alternatively, 


cos 9 = 


11-V 


u 


( 2 ) 


Our first goal in this section is to define the concept of an angle between two vectors in a general inner product space. For such 
a definition to be reasonable, we would want it to be consistent with Formula 2 when it is applied to the special case of p} and 
pf with the Euclidean inner product. Thus we will want our definition of the angle 0 between two nonzero vectors in an inner 
product space to satisfy the relationship 


cos 9 = 


MIIMI 


(3) 


However, because |cos 9\ < 1, there would be no hope of satisfying 3 unless we were assured that every pair of nonzero vectors 
in an inner product space satisfies the inequality 


Fortunately, we will be able to prove that this is the case by using the following generalization of the Cauchy-Schwarz 
inequality (see Theorem 4.1.3). 


THEOREM 6.2.1 


Cauchy-Schwarz Inequality 

If u and v are vectors in a real inner product space, then 


K«. v)| < IMINI 

(4) 


Proof We warn the reader in advance that the proof presented here depends on a clever trick that is not easy to motivate. If 
u = 0, then (u, v) = (u, u) = 0, so the two sides of 4 are equal. Assume now that u * 0- Let a — {u, u), b — 2{u, v) and 
c = (v, v), and let t be any real number. By the positivity axiom, the inner product of any vector with itself is always 










nonnegative. Therefore, 


0 < (Mi + v, t u 4- vj — (u, u}£ 2 4- 2(u, vji 4 (v, vj 
= at 2 +bt 4- c 

This inequality implies that the quadratic polynomial at? -\-bt-\- c has either no real roots or a repeated real root. Therefore, its 
discriminant must satisfy the inequality b 2 — 4ac < 0- Expressing the coefficients a, b, and c in terms of the vectors u and v 
gives 44 (u, vj — 4(u, u}(v, vj < 0, or, equivalently, 

(u, v} 2 < (ii, u}(v, v} 

Taking square roots of both sides and using the fact that (u, u} and (v, vj are nonnegative yields 

|(u, vj| < (u, u} 1/2 (v, v} 1 ^ 2 or, equivalently, |(u, vj| < ||u|| ||v|| 

which completes the proof. 

I 

For reference, we note that the Cauchy-Schwarz inequality can be written in the following two alternative forms: 

(11, V} 2 < (11, u}(v, v} (5) 

(u, v} 2 < ||u|| 2 ||v|| 2 (6) 

The first of these formulas was obtained in the proof of Theorem 6.2.1, and the second is derived from the first using the fact 
that ||n|| 2 = (u, uj and ||v|| 2 = (v, vj. 


Cauchy-Schwarz Inequality in R n 


The Cauchy-Schwarz inequality for R n (Theorem 4.1.3) follows as a special case of Theorem 6.2.1 by taking (u, v} to be the 
Euclidean inner product u - v- 

♦ 

The next two theorems show that the basic properties of length and distance that were established in Theorems 4.1.4 and 4.1.5 
for vectors in Euclidean n-space continue to hold in general inner product spaces. This is strong evidence that our definitions of 
inner product, length, and distance are well chosen. 


THEOREM 6.2.2 


Properties of Length 

If u and v are vectors in an inner product space V, and ifk is any scalar, then 

(a) IM|>0 

(b) 11 u 11 = 0 if and only if u = 0 





(c) ll*u|| = |*|IMI 

(d) ||u -(- v|| < ||u|| -I- ||v|| (Triangle inequality) 

THEOREM 6.2.3 

Properties of Distance 

If u, v, and w are vectors in an inner product space V, and if k is any scalar, then 

(a) d (u, v) > 0 

(b) d (u, v) = 0 if and only if u = v 

(c) d (u, v) = d (v, u) 

(d) d (u, v) < d (u, w) -I- d (w, v) (Triangle inequality) 

We shall prove part (d) of Theorem 6.2.2 and leave the remaining parts of Theorems Theorem 6.2.2 and Theorem 6.2.3 as 
exercises. 

Proof of Theorem 6.2.2d By definition, 


u 4 v|| 2 = (u+v, u + vj 

= (u, u} + 2(u, vj + (v, vj 

< (u, u} + 2|(u, vj| + (v, vj [Pi’operty of absolute value] 

< (u, u} h 2||u|| ||v|| + (v, v} [By (4)] 



2 


= (INI + ||v||) 


2 


Taking square roots gives ||u i v|| < ||u|| -i ||v||. 


Angle Between Vectors 


We shall now show how the Cauchy-Schwarz inequality can be used to define angles in general inner product spaces. Suppose 
that u and v are nonzero vectors in an inner product space V. If we divide both sides of Formula 6 by ||u|| 2 1| v|| 2 , we obtain 



or, equivalently, 







-1 < 


(7) 


Now if Q is an angle whose radian measure varies from 0 to tt, then cos & assumes every value between -1 and 1 inclusive 
exactly once (Figure 6.2.1). 



Thus, from 7, there is a unique angle Q such that 

„ (u, v\ 

cos0 = ——*- 


u 


and 0 < 9 < tt 


( 8 ) 


We define Q to be the angle between u and v. Observe that in p} or p^ with the Euclidean inner product, 8 agrees with the 
usual formula for the cosine of the angle between two nonzero vectors [Formula 2], 


Cosine of an Angle Between Two Vectors in /? 4 


Let j? 4 have the Euclidean inner product. Find the cosine of the angle Q between the vectors u = (4, 3, 1, — 2) and 
v= (-2, 1, 2, 3). 


Solution 


We leave it for the reader to verify that 

11u11 = ^30, ||v|| = ^1~8, and (u, vj = — 9 

so that 


cos# = 


IWIIM 



3 

2i[\5 


♦ 


Orthogonality 

Example 2 is primarily a mathematical exercise, for there is relatively little need to find angles between vectors, except in p^ 
and p} with the Euclidean inner product. However, a problem of major importance in all inner product spaces is to determine 
whether two vectors are orthogonal —that is, whether the angle between them is y = •- / 2- 

It follows from 8 that if u and v are nonzero vectors in an inner product space and Q is the angle between them, then C os 0 = 0 if 
and only if (u, v) = 0. Equivalently, for nonzero vectors we have Q = ^ j 2 if and only if j'u, vj = 0. If we agree to consider the 
angle between u and v to be ^ / 2 when either or both of these vectors is 0 , then we can state without exception that the angle 
between u and v is / 2 if and only if (u, vj = 0. This suggests the following definition. 











DEFINITION 


Two vectors u and v in an inner product space are called orthogonal if (u, vj = 0. 


Observe that in the special case where (u, vj = u v is the Euclidean inner product on R n , this definition reduces to the 
definition of orthogonality in Euclidean n-space given in Section 4.1. We also emphasize that orthogonality depends on the 
inner product; two vectors can be orthogonal with respect to one inner product but not another. 


EXAMPLE 3 Orthogonal Vectors in m 22 


If M 22 has the inner product of Example 7 in the preceding section, then the matrices 


U = 


1 0 
1 1 


and V = 


0 2 
0 0 


are orthogonal, since 


{U, V) =1(0) + 0(2) + 1(0) + 1(0) = 0 


Calculus Required 


Orthogonal Vectors in p 2 


Let p 2 have the inner product 


and let p = x and q = x 2 . Then 


<l>.q ) = f_ x 


= {p,p} 1/2 = 


= {l,q } 1/2 = 


p{x)q{x)dx 


n 1/2 


L adt \ "[/./*] = /l 

/: 


2 2 , 
x x ax 


i "i 1/2 

i> -n 


(,.,i 


x* dx = 0 


“i 

Because (p, q) = 0, the vectors P = x and q = x are orthogonal relative to the given inner product. 


In Section 4.1 we proved the Theorem of Pythagoras for vectors in Euclidean n-space. The following theorem extends this 
result to vectors in any inner product space. 


THEOREM 6.2.4 
























Generalized Theorem of Pythagoras 

Ifu and v are orthogonal vectors in an inner product space, then 

||u 4 v|| 2 = 11u11 2 4- ||v 


Proof The orthogonality of u and v implies that (u, vj = 0, so 


« 4 v 


(u 4 v, 11 + vj = ||u|| 2 4 2{u, vj 4 IIv 
l|u|| 2 +||v|| 2 


Calculus Required 


Theorem of Pythagoras in p 2 


In Example 4 we showed that p = * and q = x l are orthogonal relative to the inner product 


<* *>■/-! 

on p 2 ~ It follows from the Theorem of Pythagoras that 


p(x)q(x)dx 


IIP l-ql| 2 = lll)|| 2 + M ~" 2 


Thus, from the computations in Example 4, we have 

We can check this result by direct integration: 

111) + all 2 = (P + q, P 4 q }=j ( x + * 2 )(* + x2 ) dx 


= f x 2 dx+2j l x 2 dx 4 ^ 


x*dx 4 2/ x^dx 4 / x^dx = y + 0 4 ^ 


16 

15 


Orthogonal Complements 


♦ 


If V is a plane through the origin of with the Euclidean inner product, then the set of all vectors that are orthogonal to every 
vector in V forms the line L through the origin that is perpendicular to V (Figure 6.2.2). In the language of linear algebra we say 
that the line and the plane are orthogonal complements of one another. The following definition extends this concept to general 
inner product spaces. 





t. 





V 


Figure 6.2.2 

Every vector in L is orthogonal to every vector in V. 


DEFINITION 


Let W be a subspace of an inner product space V. A vector u in V is said to be orthogonal to W if it is orthogonal to every 
vector in W, and the set of all vectors in V that are orthogonal to W is called the orthogonal complement of W. 


Recall from geometry that the symbol [ is used to indicate perpendicularity. In linear algebra the orthogonal complement of a 
subspace W is denoted by W 1 • (read “W perp”). The following theorem lists the basic properties of orthogonal complements. 


THEOREM 6.2.5 

Properties of Orthogonal Complements 

If W is a subspace of a finite-dimensional inner product space V, then 

(a) W 1 is a subspace of V. 

(b) The only vector common to W and W 1 is 0. 

(c) The orthogonal complement ofW 1 is W; that is, (W 1 ) ' = W- 


We shall prove parts (a) and ( b ). The proof of (c) requires results covered later in this chapter, so its proof is left for the 
exercises at the end of the chapter. 


Note first that (0, wj — 0 for every vector w in W, so W 1 contains at least the zero vector. We want to show that 
W 1 is closed under addition and scalar multiplication; that is, we want to show that the sum of two vectors in W ^ is 
orthogonal to every vector in W and that any scalar multiple of a vector in W 1 is orthogonal to every vector in W. Let u and v 
be any vectors in W 1 , let k be any scalar, and let w be any vector in W. Then, from the definition of W 1 , we have (u, wj, = 0 
and (v, wj = 0. Using basic properties of the inner product, we have 












(u h v, wj = (u, wj 4- (v, wj = 0 + 0 = 0 
(£u, w} — £(u, w} — £(0) — 0 

which proves that u -i v and £u are in w 1 . 


If v is common to W and W 1 , then (v, vj, = 0, which implies that v = () by Axiom 4 for inner products. 


Remark Because W and W 1 are orthogonal complements of one another by part (c) of the preceding theorem, we shall say 
that W and W 1 are orthogonal complements. 

A Geometric Link between Nullspace and Row Space 

The following fundamental theorem provides a geometric link between the nullspace and row space of a matrix. 


THEOREM 6.2.6 


If A is an m x n matrix, then 

(a) The nullspace of A and the row space of A are orthogonal complements in R n with respect to the Euclidean inner 
product. 

(b) The nullspace of and the column space of A are orthogonal complements in R m with respect to the Euclidean 
inner product. 


Proof (a) We want to show that the orthogonal complement of the row space of A is the nullspace of A. To do this, we must 
show that if a vector v is orthogonal to every vector in the row space, then Av = Q, and conversely, that if ^y = Q, then v is 
orthogonal to every vector in the row space. 

Assume first that v is orthogonal to every vector in the row space of A. Then in particular, v is orthogonal to the row vectors rj, 
i’ 2 > . r m of A; that is, 


i’i - v = i’2 • v = — = i’m • v = 0 

But by Formula 11 of Section 4.1, the linear system Ax = 0 can t> e expressed in dot product notation as 


' l'l ■ X ' 


'o' 

1'2 * 

= 

0 



0 


so it follows from 9 that v is a solution of this system and hence lies in the nullspace of A. 


(9) 


(10) 


Conversely, assume that v is a vector in the nullspace of A, so A\ — 0- It follows from 10 that 

l’l - v - 1’2 - v - -■ = r ffl - v = 0 

But if r is any vector in the row space of A, then r is expressible as a linear combination of the row vectors of A, say 







Thus 


r = c 1 i’i +c 2 1’2 'r- + c m r m 


r' v = (c l i’l + c 2 1’2 + - + c m i' m ) - v 

= c 1 (i’l - v) + c 2 (i '2 - v) H -1 c m (r m - v) 

= 0 + 0 +-+ 0=0 

which proves that v is orthogonal to every vector in the row space of A. 


Proof (b) Since the column space of A is the row space of A 7 (except for a difference in notation), the proof follows by 
applying the result in part (a) to A 7 - 


The following example shows how Theorem 6.2.6 can be used to find a basis for the orthogonal complement of a subspace of 
Euclidean n-space. 


Basis for an Orthogonal Complement 


Let IT be the subspace of g? spanned by the vectors 

wi = (2, 2, - 1, 0, 1), w 2 = ( - 1, - 1, 2 - 3, 1), 
w 3 = (l,l, - 2 , 0 , - 1 ), w 4 = ( 0 , 0 , 1 , 1 , 1 ) 

Find a basis for the orthogonal complement of IT. 


Solution 

The space IT spanned by wj, W 2 , W 3 , and w 4 is the same as the row space of the matrix 

2-10 1 
-12-31 
1-20-1 
0 111 

and by part (a) of Theorem 6.2.6, the nullspace of A is the orthogonal complement of A. In Example 4 of Section 5.5 we 
showed that 


" - 1 " 


" -1" 

1 


0 

0 

II 

r--l 

> 

£ 

-1 

0 


0 

0 


1 


form a basis for this nullspace. Expressing these vectors in the same notation as wj, W 3 , and w 4 , we conclude that the 
vectors 

v 1= (-l, 1,0,0, 0) and v, = ( - 1, 0, - 1, 0, 1) 

form a basis for the orthogonal complement of IT. As a check, the reader may want to verify that v\ and v 2 are orthogonal to wj 
, W2> W 3 , and w 4 by calculating the necessary dot products. 


2 



0 


Summary 

We leave it for the reader to show that in any inner product space V, the zero space {0} and the entire space V are orthogonal 










complements. Thus, if A is an « x n matrix, to say that Ax. = 0 has only the trivial solution is equivalent to saying that the 
orthogonal complement of the nullspace of A is all of R n , or, equivalently, that the rowspace of A is all of Rf. This enables us 
to add two new results to the seventeen listed in Theorem 5.6.9. 

THEOREM 6.2.7 

Equivalent Statements 

If A is an nxn matrix, and if Ta Rf —* R n is multiplication by A, then the following are equivalent. 

(a) A is invertible. 

(b) = 0 has only the trivial solution. 

(c) The reduced row-echelon form of A is / . 

(d) A is expressible as a product of elementary matrices. 

( e ) Ax = b is consistent for every n x 1 matrix b. 

(f) Ax = h has exactly one solution for every n x 1 matrix b. 

(g) det(A) st 0- 

(h) The range of'p A is Rf. 

(i) p A is one-to-one. 

(j) The column vectors of A are linearly independent. 

(k) The row vectors of A are linearly independent. 

(l) The column vectors of A span R n . 

(m) The row vectors of A span Rf. 

(n) The column vectors of A form a basis for Rf. 




(o) The row vectors of A form a basis for R n . 


(p) A has rank n. 


(q) A has nullity 0. 


(r) The orthogonal complement of the nullspace of A is R n . 


(s) The orthogonal complement of the row space of A is {0}. 


This theorem relates all of the major topics we have studied thus far. 


Exercise Set 6.2 



Click here for Just Ask! 


In each part, determine whether the given vectors are orthogonal with respect to the Euclidean inner product. 

1 . 


(a) u=(-l,3,2),v=(4,2, -1) 

(b) u= (-2, -2, -2), v= (1, 1, 1) 

(c) u= (hi, u 2 , ttf), v= (0, 0, 0) 


(d) u=( —4,6, -10, 1 ),y=(2, 1, -2,9) 


(e) u= (0, 3, -2, 1), v= (5, 2, - 1, 0) 

(f) u = (a, b )> v = ( — b, a) 


Do there exist scalars k, l such that the vectors u = (2, k, 6)> v = (l, 5, 3), and w =(1,2, 3) are mutually orthogonal with 
2* respect to the Euclidean inner product? 

Let R 3 have the Euclidean inner product. Let u = (1, 1, —1) and v = (6, 7, — 15). If ||£u h v|| = 13, what is kl 


3 . 





Let R 4 have the Euclidean inner product, and let u = ( — 1, 1,0, 2)- Determine whether the vector u is orthogonal to the 
4 * subspace spanned by the vectors Wl = (0, 0, 0, 0), w 2 = (1, - 1, 3, 0), and W3 = (4, 0, 9, 2) 

Let R 2 , R 2 , and r a have the Euclidean inner product. In each part, find the cosine of the angle between u and v. 

5. 

(a) u = (1, -3), v= (2,4) 

(b) u = (_ 1, 0), v= (3, 8) 

(c) u=(-l,5,2),v=(2,4, -9) 

(d) u= (4, 1, 8), v= (1, 0, -3) 

(e) u= (1,0, l,0),v=( —3, -3, -3, -3) 

(f) u= (2, 1,7, — 1), v= (4, 0, 0, 0) 

Let p~, have the inner product in Example 8 of Section 6.1. Find the cosine of the angle between p and q. 

6 . 

(a) p = “ 1 + 5x 4- 2x 2 , q = 2 f 4x — 9x 2 


(b) p = * _ x 2 , q = 7 I 3x 4 3x 2 


Show that p = 1 — x I 2x 2 and q —2x + x 2 are orthogonal with respect to the inner product in Exercise 6. 

7. 


Let M'o have the inner product in Example 7 of Section 6.1. Find the cosine of the angle between A and B. 

8 . 



Which of the following matrices are orthogonal to A with respect to the inner product in Exercise 8? 





(a) 


-3 0 

0 2 


(b) 


1 1 
0 - 1 


(c) 


0 0 
0 0 


(d) 


2 1 
5 2 


Let p} have the Euclidean inner product. For which values of k are u and v orthogonal? 

10 . 

(a) u= (2, 1, 3), v= (1,7,*) 

(b) u = (k, k, 1)>t= (£, 5, 6) 

Let p4 have the Euclidean inner product. Find two unit vectors that are orthogonal to the three vectors u = (2, 1, —4,0). 
U * v= (- 1, -1.2. 2), and*=(3.2. 5. 4). 

In each part, verify that the Cauchy-Schwarz inequality holds for the given vectors using the Euclidean inner product. 

12 

(a) u=(3,2),v=(4, -1) 

(b) u = ( — 3, 1, 0). v= (2, -1,3) 

(c) u = ( — 4, 2, l),v=(8. -4, -2) 

(d) u = (0. -2,2, l),v=(-l, -1,1,1) 


In each part, verify that the Cauchy-Schwarz inequality holds for the given vectors. 

13. 

(a) u = ( — 2, 1) and v = (1, 0) using the inner product of Example 2 of Section 6.1 


U = 


- 1 

6 


2 

1 


and 


V = 


1 0 
3 3 


(b) 


using the inner product in Example 7 of Section 6.1 



(c) p = _ 1 4 2x I x? and q = 2 — 4x 2 using the inner product given in Example 8 of Section 6.1 


Let IE be the line in g 2 with equation y = 2x- Find an equation for W 1 • 

14. 


15. 

(a) Let IE be the plane in g 2 with equation * _ 2y — 3z — 0- Find parametric equations for W 1 . 


(b) Let IE be the line in g 2 with parametric equations 


x — 21, y — —5t, z — 4t ( — oo<£<oo) 

Find an equation for w'. 


(c) Let IE be the intersection of the two planes 


x + y + z = 0 and x—y-\-z — 0 

in g 2 . Find an equation for w 1 . 


Let 

16. 


A = 


1 2 
3 5 

1 1 


- 1 
0 
2 


2 

4 

0 


(a) Find bases for the row space and nullspace of A. 


(b) Verify that every vector in the row space is orthogonal to every vector in the nullspace (as guaranteed by Theorem 
6.2.6a). 


Let A be the matrix in Exercise 16. 

17. 

(a) Find bases for the column space of A and nullspace of 

(b) Verify that every vector in the column space of A is orthogonal to every vector in the nullspace of T ( as 
guaranteed by Theorem 6.2.6 b). 

Find a basis for the orthogonal complement of the subspace of g n spanned by the vectors. 

18. 


(a) vi = (1, -1,3 ),v 2 = (5, -4, -4),v 3 = (7, -6,2) 



(b) vi = (2,0, -l),v 2 = (4,0, -2) 


(c) V1 = (1,4, 5, 2), v 2 = (2, 1, 3, 0), v 3 = ( - 1, 3, 2, 2) 


(d) vi = (1, 4, 5, 6, 9), v 2 = (3, - 2, 1, 4, - 1), v 3 = ( - 1, 0, - 1, - 2, 


- 1), v 4 = (2, 3, 5,7, 8) 


19. 


Let V be an inner product space. Show that if u and v are orthogonal unit vectors in V, then 



Let V be an inner product space. Show that if w is orthogonal to both uj and u 2 , it is orthogonal to k\ ui -I- £ 2 u 2 for all 
20* scalars and Interpret this result geometrically in the case where V is R 3 with the Euclidean inner product. 

Let V be an inner product space. Show that if w is orthogonal to each of the vectors up u 2 , ...,Uj., then it is orthogonal to 

21. every vector in span ( Ul> U2> • 

Let ( Vli y 2 ,..., Vj.) be a basis for an inner product space V. Show that the zero vector is the only vector in V that is 

22. orthogonal to all of the basis vectors. 

Let {Vp w 2 , w^} be a basis for a subspace W of V. Show that W 1 consists of all vectors in V that are orthogonal to 

23. every basis vector. 


Prove the following generalization of Theorem 6.2.4. If vp v 2 , ..., v r are pairwise orthogonal vectors in an inner product 

24. space V, then 


||vi +v 2 + -v,|| =||vi|| +||v 2 || 4-h ||v r 


Prove the following parts of Theorem 6.2.2: 

25. 


(a) part (a) 


(b) part {b) 


(c) part (c) 


Prove the following parts of Theorem 6.2.3: 

26 . 


(a) part (a) 


(b) part ( b ) 


(c) part (c) 



(d) part ( d ) 


Prove: If u and v are n x 1 matrices and A is an n x n matrix, then 

27. 7 

n-i r-n n-i r-n n-i r-p 

(y T A 7 Aa) < (n T A T Aa) (v T A 7 Av) 

Use the Cauchy-Schwarz inequality to prove that for all real values of a, b, and Q, 

28 

(a cos# I £sm#) 2 <a 2 I b l 

Prove: If wp W 2 , w n are positive real numbers and if u = (u\, uj, - u M ) ar| d v = (vi, V 2 ,v M ) are any two vectors in 
29- then 

|wi wi vi -\ W2^2^2 + ■■■+ 

2 2 2 2 2 2 ^ 

<(W|U[ 4 W2I<2 f-“ + w M u M ) (W1VJ+W2V2H-1- 

Show that equality holds in the Cauchy-Schwarz inequality if and only if u and v are linearly dependent. 

30. 


Use vector methods to prove that a triangle that is inscribed in a circle so that it has a diameter for a side must be a right 
31. triangle. 



Hint Express the vectors and gQ in the accompanying figure in terms of u and v. 

With respect to the Euclidean inner product, the vectors u = (1, ^3) and v = ( — 1, \[3) have norm 2, and the angle 
32* between them is 60°. (see the accompanying figure). Find a weighted Euclidean inner product with respect to which u and 
v are orthogonal unit vectors. 



33. (For Readers Who Have Studied Calculus) 

Let f (x) and g(v) be continuous functions on [0, 1]. Prove: 





QQ 


(a) 

r 1 

2 

/A n 

/A 


/ f(x)g(.x)dx 

,k 

< 

/ f 2 (x)dx 

Jo 

/ g 2 (x)dx 

Jo 


(b) 


1/2 


1/2 

f 1 


/ [/(. x ) + g( x )] 2 d* 
Jo 

< 

j ^ f 2 {x)dx 

A 

/ g 2 (x)d* 
Jo 


Hint Use the Cauchy-Schwarz inequality. 


34. (For Readers Who Have Studied Calculus) 

Let C[ 0, ttI have the inner product 

{f,g) = jjf(x)g(x)dx 

and let f n = cos nx {n = 0, 1,2,...)- Show that if l : ^ /, then f ^ and f j are orthogonal with respect to the given inner 
product. 


iscussion 
iscovery 


35. 


(a) Let W be the line y = x in an ^-coordinate system in p 2 . Describe the subspace W 


(b) Let Wbt the y-axis in an xyz-coordinate system in p}. Describe the subspace W 1 • 

(c) Let W be the yz-plane of an xyz-coordinate system in j? 3 . Describe the subspace W 1 • 


Let j4x = 0 t> e a homogeneous system of three equations in the unknowns x, y, and z. 

36. 


(a) If the solution space is a line through the origin in p > 3 , what kind of geometric object is 
the row space of A? Explain your reasoning. 

(b) If the column space of A is a line through the origin, what kind of geometric object is the 
solution space of the homogeneous system A T x. = 0? Explain your reasoning. 

(c) If the homogeneous system A 7 x = 0 has a unique solution, what can you say about the 
row space and column space of A? Explain your reasoning. 


37 . 


Indicate whether each statement is always true or sometimes false. Justify your answer by givinj 
a logical argument or a counterexample. 
















(a) If V is a subspace of R n and W is a subspace of V, then 1 is a subspace of V 1 • 


(b) ||u -f v 4- w|| < 11u11 + ||v|| 4- ||w|| for all vectors u, v, and w in an inner product space. 


(c) If u is in the row space and the nullspace of a square matrix A, then u = 0- 


(d) If u is in the row space and the column space of an n x n matrix A, then u = 0- 


38. 


T T 

Let Mj2 ' iave inner product /' U, V) = tx(JJ V) =Xx(V U) that was defined in Example 7 
of Section 6.1. Describe the orthogonal complement of 


(a) the subspace of all diagonal matrices 


(b) the subspace of symmetric matrices 


Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 



6.3 

ORTHONORMAL BASES; 

GRAM-SCHMIDT 

PROCESS; 

QR-DECOM POSITION 


In many problems involving vector spaces, the problem solver is free to 
choose any basis for the vector space that seems appropriate. In inner 
product spaces, the solution of a problem is often greatly simplified by 
choosing a basis in which the vectors are orthogonal to one another. In this 
section we shall show how such bases can be obtained. 


DEFINITION 


A set of vectors in an inner product space is called an orthogonal set if all pairs of distinct vectors in the set are 
orthogonal. An orthogonal set in which each vector has norm 1 is called orthonormal. 


An Orthogonal Set in /? 3 


Let 


ui = ( 0 , 1 , 0 ), u 2 = ( 1 , 0 , 1 ), 113 = ( 1 , 0 , — 1 ) 

and assume that p-' has the Euclidean inner product. It follows that the set of vectors S= { 111 , 112 , 113 } is orthogonal since 
(ui, u 2 ) = (ui, 113 } = ( 112 , 113 } = 0. 

♦ 


If v is a nonzero vector in an inner product space, then by part (c) of Theorem 6.2.2, the vector 

1 


-v 


has norm 1, since 


1 v 


1 

IMI 


IMI 


v = ■ 


1 


V 


Ml = 1 


The process of multiplying a nonzero vector v by the reciprocal of its length to obtain a unit vector is called normalizing v. 
An orthogonal set of nonzero vectors can always be converted to an orthonormal set by normalizing each of its vectors. 


Constructing an Orthonormal Set 


The Euclidean norms of the vectors in Example 1 are 

ll«l 11 = 1. M = i/ 2 . II 113 II = 1/2 

Consequently, normalizing ui, 112 , and 113 yields 






















vi = - ii ^ = C°, 1,0), T 2 = -jpV = -7=. 0. -7= 1 


v, = _33_ = 

3 ll«3ll 




fi' ’ 

We leave it for you to verify that the set S = {y 1; y 2 , v 3 } is orthonormal by showing that 

(vi,V2} = (vi,V3} = (v 2 ,v 3 } = 0 and ||vi|| = ||v 2 || = ||v 3 | 


= 1 


In an inner product space, a basis consisting of orthonormal vectors is called an orthonormal basis, and a basis consisting of 
orthogonal vectors is called an orthogonal basis. A familiar example of an orthonormal basis is the standard basis for 
with the Euclidean inner product: 

i= (1,0,0), j= (0,1,0), k= (0,0,1) 

This is the basis that is associated with rectangular coordinate systems (see Figure 5.4.4). More generally, in R n with the 
Euclidean inner product, the standard basis 

ei = (1, 0, 0, 0), e 2 = (0, 1, 0,.... 0),.... pj, = (0, 0, 0,.... 1) 


is orthonormal. 


Coordinates Relative to Orthonormal Bases 

The interest in finding orthonormal bases for inner product spaces is motivated in part by the following theorem, which 
shows that it is exceptionally simple to express a vector in terms of an orthonormal basis. 


THEOREM 6.3.1 


If S— (vi, v' 2 ,..., v H ) i s an orthonormal basis for an inner product space V, and u is any vector in V, then 

u = (u, vi) vi 4- { 11 , v 2 )v 2 + - 4- {u, v H )v H 


Proof Since S= (vj, v 2 ,..., v H ) is a basis, a vector u can be expressed in the form 

u = £1 vi 4- k 2 v 2 4-... 4- k n v H 

We shall complete the proof by showing that £,• = (u, v 3 ) for i = 1, 2, ..., n. For each vector v 3 in S, we 
have 

(U> V 3 } = (jfci VI -I- k 2 v 2 4- - 4 k n v„, v 3 } 

= *l(vi, v 3 } 4- i 2 (v 2 , v 3 J 4- 

Since s= (vi, v 2 ,..., v H } is an orthonormal set, we have 

(v 3 , v 3 } = ||v 3 || 2 = 1 and (vj, v 3 } = 0 if j*i 
Therefore, the above expression for (u, v 3 \ simplifies to 

(u, Vj) = kj 


Using the terminology and notation introduced in Section 5.4, the scalars 

(u,v 2 },..., ( 11 , V H } 






in Theorem 6.3.1 are the coordinates of the vector u relative to the orthonormal basis S= {vi, V 2 ,..., v H }, and 

(v)s= C( u > v l}> ( u > v 2 }> («. V„J) 

is the coordinate vector of u relative to this basis. 


Coordinate Vector Relative to an Orthonormal Basis 


Let 


v i — (0, 1, 0), v 2 =(—4 0 ,|), 


v 3 = 



) 


It is easy to check that S = {vq, v 2 , v 3 } is an orthonormal basis for with the Euclidean inner product. Express the vector 
«= (1.1.1) as a linear combination of the vectors in S, and find the coordinate vector ( u ) s . 


Solution 


1 7 

(u, vi J = 1, (u, v 2 J = - - and (u, v 3 J = - 


Therefore, by Theorem 6.3.1 we have 

that is, 


1 7 

« = vi - yv 2 4 yv 3 


( 1 , 1 . 1 ) = ( 0 , 1 , 0 ) - i( - i 0 , |] + ^f|, 0 , | 


The coordinate vector of u relative to S is 

( u ) s = C{ u > vi}, (u, v 2 }, (u, v 3 J) = (1, 


1 1 
5’ 5 


♦ 


Remark The usefulness of Theorem 6.3.1 should be evident from this example if we remember that for nonorthonormal 
bases, it is usually necessary to solve a system of equations in order to express a vector in terms of the basis. 


Orthonormal bases for inner product spaces are convenient because, as the following theorem shows, many familiar formulas 
hold for such bases. 


THEOREM 6.3.2 


If S is an orthonormal basis for an n-dimensional inner product space, and if 


then 


(v)s= C«i.« 2 .— -«n) and (y)s= (vi.v 2 ,...,v„) 


(a) 


u 


= / 


a? + uj + ■ 


■ H Un 







Observe that the right side of the equality in part (a) is the norm of the coordinate vector ( u ) ^ with respect to the 
Euclidean inner product on R ”, and the right side of the equality in part (c) is the Euclidean inner product of ( u ) ^ and ( v ) $■ 
Thus, by working with orthonormal bases, we can reduce the computation of general norms and inner products to the 
computation of Euclidean norms and inner products of the coordinate vectors. 


Calculating Norms Using Orthonormal Bases 


If R 2 has the Euclidean inner product, then the norm of the vector u = ( 1 , 1 ) is 

Hull = (u ■ u) 1/2 = f 2 1 1 2 + 1 2 = t/3 

However, if we let R 3 have the orthonormal basis S in the last example, then we know from that example that the coordinate 
vector of u relative to S is 

The norm of u can also be calculated from this vector using part (a) of Theorem 6.3.2. This yields 

'"'-i 


Coordinates Relative to Orthogonal Bases 

If S— {vi, \’ 2 ,..., v H } is an orthogonal basis for a vector space V, then normalizing each of these vectors yields the 
orthonormal basis 


S' = l^ 

IhT | 


2l2_ 

l|v 2 | 


v, 


Thus, if u is any vector in V, it follows from Theorem 6.3.1 that 


4- u, 


u, 


VI 

\ VI 


) llvill 


v 2 

i v 2 

IIV2II 

) II V2 II 


-|--h ll. 


which, by part (c) of Theorem 6.1.1, can be rewritten as 


v w 1 

[ v„ 

l|v H || J 

1 l|v„|| 

-v n 



IhTi 


W2\ 


v. 


( 1 ) 


This formula expresses u as a linear combination of the vectors in the orthogonal basis S. Some problems requiring the use of 
this formula are given in the exercises. 


It is self-evident that if vj, \ r 2 , and V 3 are three nonzero, mutually perpendicular vectors in R 2 , then none of these vectors lies 
in the same plane as the other two; that is, the vectors are linearly independent. The following theorem generalizes this result. 





















THEOREM 6.3.3 


If S= (vj, V 2 , v vj} is an orthogonal set of nonzero vectors in an inner product space, then S is linearly independent. 


Proof Assume that 


£l vi f k 2 \ r 2 + - H k n v„ — 0 


( 2 ) 


To demonstrate that s= (vi, V 2 ,..., v„} is linearly independent, we must prove that ^ = k 2 = = k n = 0 


For each \q in S, it follows from 2 that 

(*1 V 1 + k 2 v 2 + -k n v„, v,) = (0, V; ) = 0 


or, equivalently, 


*l(vi, v,} 4- k 2 {v 2 , v, } + ■*■ + k„{Y„, v,-} = 0 

From the orthogonality of S it follows that !v^ v,-J = 0 when j * i, so this equation reduces to 

^(Vj, v, } = 0 


Since the vectors in S are assumed to be nonzero, (v, , v, J ^ 0 by the positivity axiom for inner products. Therefore, lr- = Q. 
Since the subscript i is arbitrary, we have k\=k 2 = = 0» thus S is linearly independent. 


Using Theorem 6.3.3 


In Example 2 we showed that the vectors 


1 


1 


vi = ( 0 , 1 , 0 ), v 2 = H=, 0 ,—H, and v 3 = 


f ' ft, 


_L o --L.' 
f2- ■ f 2) 


form an orthonormal set with respect to the Euclidean inner product on pf. By Theorem 6.3.3, these vectors form a linearly 
independent set, and since pf is three-dimensional, S= (vi, V 2 , v 3 } is an orthonormal basis for pf by Theorem 5.4.5. 

♦ 


Orthogonal Projections 

We shall now develop some results that will help us to construct orthogonal and orthonormal bases for inner product spaces. 

In p 2 or pf with the Euclidean inner product, it is evident geometrically that if IT is a line or a plane through the origin, then 
each vector u in the space can be expressed as a sum 

u = wq -I- W2 

where wq is in W and W 2 is perpendicular to W (Figure 6.3.1). This result is a special case of the following general theorem 
whose proof is given at the end of this section. 







(fJJ 



THEOREM 6.3.4 


Projection Theorem 

IfW is a finite-dimensional subspace of an inner product space V, then every vector u in V can be expressed in exactly 
one way as 


u = wj + W 2 


(3) 


where wj is in W and w 2 is in w 1 . 


The vector wi in the preceding theorem is called the orthogonal projection of u on W and is denoted by proj ^u- The vector 
W 2 is called the component of u orthogonal to W and is denoted by proj M , i u. Thus Formula 3 in the Projection Theorem can 
be expressed as 


u = proj^u -1- proj^i. u 


(4) 


Since w 2 = u — wi it follows that 
so Formula 4 can also be written as 


proj^i u = u —proj^u 


u = propf/u + (u - proijf/u) 


(5) 


(Figure 6.3.2). 



Figure 6.3.2 










The following theorem, whose proof is requested in the exercises, provides formulas for calculating orthogonal projections. 


THEOREM 6.3.5 


Let W be a finite-dimensional subspace of an inner product space V. 


( a ) If {vi,V2, -- 

^ v j.} is an orthonormal basis for W, and u is any vector in V, then 



projff/u = <n, yi) vi -1 <11, v 2 )v2 + - -1 (it, Vj.)v,. 

(6) 

(b) If {vi,V2, .. 

v r } is an orthogonal basis for IT, and u is any vector in V, then 



<u, vi) {u,V 2 } , , (u, 

proj^/u = -vi+ - v 2 1-1 , v, 

llvill 2 II V 2 II 2 I|V,|| 2 

(7) 


Calculating Projections 


Let pf have the Euclidean inner product, and let IT be the subspace spanned by the orthonormal vectors = (0, 1, 0) and 
V 2 = | — %, 0, -jj. From 6 the orthogonal projection of u = (1, 1, 1) on IT is 

projflru = (u, vi }vi + (u, v 2 }v 2 

= (l)( 0 ,l, 0 ) + (-I){-i 0 ,f) 

= (± 1 

\ 25 ’ ’ 25 } 

The component of u orthogonal to IT is 

proj^i u = u —projjpu= ( 1 , 1 , 1 ) - 1 , - 0 , ||j 

Observe that proj^ i u is orthogonal to both vj and y 2 , so this vector is orthogonal to each vector in the space IT spanned by 
vj and v 2 , as it should be. 

♦ 


Finding Orthogonal and Orthonormal Bases 

We have seen that orthonormal bases exhibit a variety of useful properties. Our next theorem, which is the main result in this 
section, shows that every nonzero finite-dimensional vector space has an orthonormal basis. The proof of this result is 
extremely important, since it provides an algorithm, or method, for converting an arbitrary basis into an orthonormal basis. 


THEOREM 6.3.6 


Every nonzero finite-dimensional inner product space has an orthonormal basis. 









Let Vbe any nonzero finite-dimensional inner product space, and suppose that (up 112 ,u H } is any basis for V. It 
suffices to show that V has an orthogonal basis, since the vectors in the orthogonal basis can be normalized to produce an 
orthonormal basis for V. The following sequence of steps will produce an orthogonal basis (y 1; y 2 , v„} for V. 



Jorgen Pederson Gram (1850-1916) was a Danish actuary. Gram's early education was at village schools supplemented 
by private tutoring. After graduating from high school, he obtained a master's degree in mathematics with specialization in 
the newly developing modern algebra. Gram then took a position as an actuary for the Hafnia Life Insurance Company, 
where he developed mathematical foundations of accident insurance for the company Skjold. He served on the Board of 
Directors of Hafnia and directed Skjold until 1910, at which time he became director of the Danish Insurance Board. 
During his employ as an actuary, he earned a Ph.D. based on his dissertation “On Series Development Utilizing the Least 
Squares Method.” It was in this thesis that his contributions to the Gram- Schmidt process were first formulated. Gram 
eventually became interested in abstract number theory and won a gold medal from the Royal Danish Society of Sciences 
and Letters for his contributions to that field. However, he also had a lifelong interest in the interplay between theoretical 
and applied mathematics that led to four treatises on Danish forest management. Gram was killed one evening in a bicycle 
collision on the way to a meeting of the Royal Danish Society. 


Step 1. Let yj — up 

Step 2. As illustrated in Figure 6.3.3, we can obtain a vector V 2 that is orthogonal to by computing the component of 
112 that is orthogonal to the space W\ spanned by vp We use Formula 7: 



v 2 = «2 - «2 = «2 - ^-9 V 1 

llvill 2 


Of course, if V2 = 0> then V 2 is not a basis vector. But this cannot happen, since it would then follow from the preceding 
formula for v 2 that 







The preceding step-by-step construction for converting an arbitrary basis into an orthogonal basis is called the 
Gram-Schmidt process. 


Using the Gram-Schmidt Process 


Consider the vector space p} with the Euclidean inner product. Apply the Gram-Schmidt process to transform the basis 
vectors = ( 1 , 1 , 1 ), u 2 = ( 0 , 1 , 1 ), uj = ( 0 , 0 , 1 ) into an orthogonal basis { vj, V 2 , Y 3 }; then normalize the orthogonal 
basis vectors to obtain an orthonormal basis (q 1; q 2; < 13 } • 


Solution 

Stepl. v'i = ui = (1, 1, 1) 


Step 2. 


( u 2 > V 1 ) 

v 2 = «2 - P ro JWi «2 = «2 - --r-vi 

II vi || 2 




V3 = »3 - P r °J ^ 2 "3 = u 3 


(ii3’Vi) Vi (H3.V2) 


I vi | 


|V2l 


Ste p3. = (o, i, i) _ l(i, 1.1) - ILL ( _ 1 I I 


= f 0 -1 1 
2 ’ 2 


Thus 


vi = 0-1-1)- v 3=( 0 '“2'2 

form an orthogonal basis for p\ The norms of these vectors are 

llvill = |/3, || v 2 1| = ||v 3 || = -j= 


so an orthonormal basis for p} is 

_ vi _ \ 1 1 1 


<11 


VIII ft’ ftf 

<13 = 


<12 


_ >'2 _ 
l|v 2 || 


2 J_L 

{& ’ je’ jej 


0 --L _L- 

l|v 3 || ’ "" 


1/2' 


♦ 












Erhardt Schmidt ( 1876 - 1959 ) was a German mathematician. Schmidt received his doctoral degree from Gottingen 
University in 1905, where he studied under one of the giants of mathematics, David Hilbert. He eventually went to teach 
at Berlin University in 1917, where he stayed for the rest of his life. Schmidt made important contributions to a variety of 
mathematical fields but is most noteworthy for fashioning many of Hilbert's diverse ideas into a general concept (called a 
Hilbert space), which is fundamental in the study of infinite-dimensional vector spaces. Schmidt first described the 
process that bears his name in a paper on integral equations published in 1907. 


Remark In the preceding example we used the Gram-Schmidt process to produce an orthogonal basis; then, after the entire 
orthogonal basis was obtained, we normalized to obtain an orthonormal basis. Alternatively, one can normalize each 
orthogonal basis vector as soon as it is obtained, thereby generating the orthonormal basis step by step. However, this 
method has the slight disadvantage of producing more square roots to manipulate. 


The Gram-Schmidt process with subsequent normalization not only converts an arbitrary basis (uj, 112 ,u H } into an 
orthonormal basis (q^ q 2 i q H } but does it in such a way that for k >2 the following relationships hold: 


{91. Q2 >qjc) is an orthonormal basis for the space spanned by (uj, 112 ,u^} • 


(\ir is orthogonal to the space spanned by (m, 112 ,u^_i} • 


We omit the proofs, but these facts should become evident after some thoughtful examination of the proof of Theorem 6.3.6. 

QK-Decomposition 

We pose the following problem. 


Problem If A is an m x n matrix with linearly independent column vectors, and if Q is the matrix with orthonormal column 
vectors that results from applying the Gram-Schmidt process to the column vectors of A, what relationship, if any, exists 
between A and Q1 


To solve this problem, suppose that the column vectors of A are uj, 112, ..., u M and the orthonormal column vectors of Q are 

<11, <i2> •• •> q M ; thus 

A= [ui|u2h|u H ] and Q= [qi |q2|-|q M ] 



It follows from Theorem 6.3.1 that ui, « 2 , ..., u H are expressible in terms of the vectors qi, q 2 , ..., q H as 

ui - (ui, qi}qi + (ui, q2}q2 +"+ («1. q«}q« 

U 2 = ( 112 , qi}qi f (ll 2 , + “ + (« 2 . q«}q« 


«m = (% qi}qi +(% q2}q2 + -“ + («H. q H }q« 

Recalling from Section 1.3 that the /th column vector of a matrix product is a linear combination of the column vectors of the 
first factor with coefficients coming from the yth column of the second factor, it follows that these relationships can be 
expressed in matrix form as 


[ui|U 2 |-‘-|u„] = [qi|q 2 h|q M ] 


(HI. qi} 

( u 2 > qi} ■■ 

■■ {«». qi} 

(*i. q2} 

(U2, q2} " 

■■ (««. q2} 

(ui. q«} 

(«2. q«} ■■ 

'■ ( u h. q«} 


or more briefly as 


A = QR 


( 8 ) 


However, it is a property of the Gram-Schmidt process that for j>2, the vector qy is orthogonal to ui, 112 , 
entries below the main diagonal of R are zero, 

{«i.qi} {«2>qi} ™ (n B , qi} 

R _ 0 {«2-q2} - q2} 

0 0 ■■■ (u„, q H J 


..., thus, all 


(9) 


We leave it as an exercise to show that the diagonal entries of R are nonzero, so R is invertible. Thus Equation 8 is a 
factorization of A into the product of a matrix Q with orthonormal column vectors and an invertible upper triangular matrix 
R. We call Equation 8 the QR-decomposition of A. In summary, we have the following theorem. 


THEOREM 6.3.7 


Q/7-Decomposition 

If A is an mxn matrix with linearly independent column vectors, then A can be factored as 

A = QR 

where Q is an mxn matrix with orthonormal column vectors, and R is an « x « invertible upper 
triangular matrix. 


Remark Recall from Theorem 6.2.7 that if A is an N x « matrix, then the invertibility of A is equivalent to linear 
independence of the column vectors; thus, every invertible matrix has a ^-decomposition. 


Qft-Decomposition of a 3 x 3 Matrix 


Find the ^-decomposition of 









Solution 


A = 


1 0 0 
1 1 0 
1 1 1 


The column vectors of A are 



'f 


'O' 


'O' 

U1 = 

1 

, «2 = 

1 

, 113 = 

0 


1 


1 


1 


Applying the Gram-Schmidt process with subsequent normalization to these column vectors yields the orthonormal vectors 
(see Example 7) 



1 / ifl 


- 2/^6 


o ' 

11 = 

1 / lf3 

> 12 = 

1 / {e 

. 13 = 

-If {2 


1 / /3 


1 / {e 


1 / {2 


and from 9 the matrix R is 



(«l. 11 J ( 112 . 11 } { 113 . 11 } 


1 

U-j 

To] 

[-0 

To] 

Tn 
_1 

R = 

0 { 112 . 12 } { 113 . 12 } 

= 

0 2 f{l 1 f{l 


0 0 (U 3> q 3 } 


0 0 1 / /2 


Thus the (^-decomposition of A is 


'1 0 o ' 


1 / 

-21 {6 

0 

31 \[3 

2/^3 

1 / 

1 1 0 

= 

1 / 

1 / 

- 1 / {2 

0 

2 ! {1 

1 / {e 

1 1 1 


1 / ft 

1 / \ j & 

1 / {2 

0 

0 

1 / /2 


A Q R 


The Role of the QK-Decomposition in Linear Algebra 


In recent years the (^-decomposition has assumed growing importance as the mathematical foundation for a wide variety of 
practical numerical algorithms, including a widely used algorithm for computing eigenvalues of large matrices. Such 
algorithms are discussed in textbooks that deal with numerical linear algebra. 


Additional Proof 


Proof of Theorem 6.3.4 There are two parts to the proof. First we must find vectors wj and W 2 with the stated properties, 
and then we must show that these are the only such vectors. 

By the Gram-Schmidt process, there is an orthonormal basis ( Vli v 2 i y H ) for W. 

Let 


wi = («. V 1 } V 1 + (u, V 2 )V 2 + - 4- (u, v„}v„ 


( 10 ) 


and 


























w 2 = u - W! 


(ID 


It follows that Yir'i | W 2 = wi 4 - (u — wi) = u> so it remains to show that wj is in W and W 2 is orthogonal to W. But wj lies in 
W because it is a linear combination of the basis vectors for W. To show that W 2 is orthogonal to W, we must show that 
(w 2 , wj = 0 for every vector w in W. But if w is any vector in W, it can be expressed as a linear combination 

w = ^ vi -I- k 2 V 2 4 - T k n v H 

of the basis vectors vi, V 2 , •••, v H . Thus 

(W 2 , W} = (11 - Wl, w) = (u, w} - (wi, w) ( 12 

But 

(u, wj = (u, ,ti vi 4 k 2 v '2 + - 4 k n v H J 

= *l{™. V 1} + * 2 ( 11 . v 2 } 4 - I- k„{ u, v M J 

and by part (c) of Theorem 6.3.2, 

(wi, w} = (u, vi yti f (u, V 2}^2 + - + (u, 

Thus (u, wj and (wi, wj are equal, so 12 yields (w 2 , wj = 0, which is what we want to show. 


To see that 10 and 11 are the only vectors with the properties stated in the theorem, suppose that we can also write 


u = w , i -I- w 7 2 


(13) 


where w^i is in W and I s orthogonal to W. If we subtract from 13 the equation 

u = wj 4 w‘2 

we obtain 

0 = (w'i - wq) + (w / 2 -w 2 ) 
or 


Wl -w'l = wS - V‘2 


(14) 


Since and w^ are orthogonal to W, their difference is also orthogonal to W, since for any vector w in W, we can write 

{w, w ; 2 - W 2 } = (w, w' 2 ] - (w, W 2 } = 0 - 0 = 0 

But w 7 2 — W 2 is itself a vector in W, since from 14 it is the difference of the two vectors wj and w r i that lie in the subspace W. 
Thus, 2 — W 2 must be orthogonal to itself; that is, 

fw'2 - w'2, w'2 - W2} = 0 

But this implies that w^ — W 2 = 0 by Axiom 4 for inner products. Thus w '2 = w 2 > and by 14, w'i = wp 


Exercise Set 6.3 



Click here for Just Ask! 


1 . 


Which of the following sets of vectors are orthogonal with respect to the Euclidean inner product on p} 





(a) (0,1), (2,0) 


(b) (- 1 / J2, 1 /1/2)’ (1 /1/2, 1 /1/2) 

(c) (_ 1 / j/ 2 , _ 1 / ^ 2 ), (1 / ^ 2 , 1 / ^ 2 ) 

(d) (0, 0), (0, 1) 


Which of the sets in Exercise 1 are orthonormal with respect to the Euclidean inner product on 

2 . 


Which of the following sets of vectors are orthogonal with respect to the Euclidean inner product on 

3. 


W (/T* / 2MJ3 ' / 3 - “ /b)’ (“ / 2 -°- k 


(b) (2 _2 n [2 1 _2W1 2 2) 

\3’ 3’ 3J’ 13’ 3’ 3/ i3’ 3’ 3} 


<C) 0.0,0), (o,-L,-L), (0,0,1) 


(d) I 1 1 


1 1 


{l’ {s’ {6 )’ {{2’ {2’ 


Which of the sets in Exercise 3 are orthonormal with respect to the Euclidean inner product on # 3 ? 

4. 


Which of the following sets of polynomials are orthonormal with respect to the inner product on discussed in Example 
5* 8 of Section 6.1? 


(a) + + 1 X 1 


(b) , -Lx | -U 2 2 
’ p. fe ’* 







Which of the following sets of matrices are orthonormal with respect to the inner product on M 22 discussed in Example 7 
6 * of Section 6.1? 


(a) 

'1 O' 


'» 3 " 


'» f 


f 


_° 0_ 


1 2 

3 3 


2 1 

3 3 


2 2 

3 3 


(b) 


'1 O' 


'0 f 


'0 O' 


'0 0 

_° 0_ 

? 

.0 °. 

7 

_1 1_ 

7 

_1 -1_ 


Verify that the given set of vectors is orthogonal with respect to the Euclidean inner product; then convert it to an 
7. orthonormal set by normalizing the vectors. 


(a) (- 1 , 2 ), ( 6 , 3) 


(b) (1,0, - 1 ), (2, 0,2), (0,5,0) 


(c) 


(l 1 n 

\5’ 5’ 5 f 


1 1 0 ] (1 1 - 1 ) 

2’ 2’ f 13’ 3’ 3} 


Verify that the set of vectors {(1, 0), (0, 1)} is orthogonal with respect to the inner product /u, v\ —4u\v\ H U 2 V 2 on p 2 ; 
°* then convert it to an orthonormal set by normalizing the vectors. 


9. 


Verify that the vectors vi = | — j, j, OJ, v r 2 = j, OJ, V 3 = (0, 0, 1) form an orthonormal basis for with the 
Euclidean inner product; then use Theorem 6.3.1 to express each of the following as linear combinations of vi, V 2 , and V 3 . 


(a) (1, -1,2) 


(b) (3, -7,4) 


(c) 


T _3 5 3 

?, 7 , ? j 


Verify that the vectors 

V1 = (1, - 1, 2, - 1), v 2 = (-2, 2, 3, 2), v 3 = (1, 2, 0, - 1), v 4 =(l,0, 0, 1) 

form an orthogonal basis for p 4 with the Euclidean inner product; then use Formula 1 to express each of the following 
linear combinations of vi, \ r 2, V3, and v 4 . 




(a) (1, 1, 1, 1) 


(b) (/2, -3/2, 5/2, -^2) 

(c) / 1 2 _1 4) 

l 3’3' 3’ 3 ) 


In each part, an orthonormal basis relative to the Euclidean inner product is given. Use Theorem 6.3.1 to find the 
11 * coordinate vector of w with respect to that basis. 


(a) 


w=(3,7);«i 



«2 



(b) 


»■=(■= 1 , 0 , 2 ); ui = 






(12 2 
p 3’ 3 


\ 


12 . 


Let p} have the Euclidean inner product, and let p = ( Wl , W 2 ) be the orthonormal basis with vq = 




(a) Find the vectors u and v that have coordinate vectors (u) ^ = (1, 1) and ( v ) s = ( — 1,4). 

(b) Compute ||u||, ^( u , v)> and ( 11 , v) by applying Theorem 6.3.2 to the coordinate vectors ( u ) s and ( v ) then check 
the results by performing the computations directly on u and v. 


Let Pc’ have the Euclidean inner product, and let S = (vq, W2, W3} be the orthonormal basis with vq = |o, — ^ 

w 2 =(L0, 0 ),and W3 = (o, ^ |j- 


(a) Find the vectors u, v, and w that have the coordinate vectors ( u ) s = ( _ 2, 1,2), (v) s = (3, 0, — 2), and 
(w)<v=(5, -4,1). 


(b) Compute ||v||, d ( 11 , w), and (w, vj by applying Theorem 6.3.2 to the coordinate vectors ( u ) ( v ) o, and ( w ) q 

then check the results by performing the computations directly on u, v, and w. 



In each part, S represents some orthonormal basis for a four-dimensional inner product space. Use the given information 
14. to find 11 u 11 , ||v —w||, ||v4 w||, and (v, wj. 

(a) Cn) 5 =(-l,2,l,3),(v) ff =C0, - 3, 1, 5), (w) s = ( - 2 , -4,3,1) 

(b) (u)^=(0,0, -1, -1), (v)^=(5,5, -2, -2),(w) J =(3,0, -3,0) 


15. 


(a) Show that the vectors V1 = ( 1 , -2,3, -4),v 2 = ( 2 , 1 , -4, - 3), v 3 = ( - 3, 4, 1 , - 2 ), and V4 = (4, 3, 2 , 1 ) 
form an orthogonal basis for p 4 with the Euclidean inner product. 


(b) Use 1 to express u = (— 1 , 2 , 3 , 7 ) as a linear combination of the vectors in part (a). 


Let p 2 have the Euclidean inner product. Use the Gram-Schmidt process to transform the basis (m, u 2 ) into an 
16* orthonormal basis. Draw both sets of basis vectors in the xy-plane. 


(a) u! = (l, -3), u 2 =(2,2) 


(b) ii! = (1,0), u 2 = (3, -5) 


Let Pc’ have the Euclidean inner product. Use the Gram-Schmidt process to transform the basis ( Ul , U 2 , 113} into an 
1 ^* orthonormal basis. 


(a) ii! = (1, 1, 1), 112 = ( - 1, 1, 0), 113 = (1, 2, 1) 

(b) iii = (1, 0, 0), u 2 = (3, 7, - 2), u 3 = (0, 4, 1) 

Let p 4 have the Euclidean inner product. Use the Gram-Schmidt process to transform the basis (m, 112 , 113 , 114 } into an 
18* orthonormal basis. 

m = (0,2,1,0), U 2 = (l,-1,0,0), U 3 = (l,2,0,-1), ii4=O,0,0,l) 

Let p} have the Euclidean inner product. Find an orthonormal basis for the subspace spanned by (0, 1, 2), (-1, 0, 1), (-1, 
19 - 1,3). 

Let p} have the inner product /u, v\ = vi -I- 2u 2 I 3 u 3 v 3 . Use the Gram-Schmidt process to transform 

20 '' 7 

Ul = ( 1 , 1 , 1 ), U 2 = ( 1 , 1 , 0 ), 113 = ( 1 , 0 , 0 ) into an orthonormal basis. 



The subspace of j ? 3 spanned by the vectors uj = 0, — and 113 = (0, 1, 0) is a plane passing through the origin. 

Express w = (1, 2, 3) in the form w — W1 4 . W2 , where wj lies in the plane and W 2 is perpendicular to the plane. 

Repeat Exercise 21 with m = (1, 1, 1) and u 2 = ( 2 , 0 , - 1 ). 


Let 2? 4 have the Euclidean inner product. Express w = ( — 1, 2, 6 , 0) in the form w — W1 _|_ W9 , where w\ is in the space 
W spanned by m = ( — 1, 0, 1,2) and u 2 = (0, 1,0, 1), and W 2 is orthogonal to W. 


Find the ^-decomposition of the matrix, where possible. 

24. 


(a) 


1 - 1 

2 3 


(b) 


1 2 
0 1 
1 4 


(c) 


1 1 

-2 1 

2 1 


(d) 


1 0 2 
0 1 1 
1 2 0 


(e) 


1 2 1 
1 1 1 

0 3 1 


(f) 


1 

0 

1 

-1 1 

1 

1 

0 

1 

-1 1 

1 


Let { vj, V 2 , v’ 3 } be an orthonormal basis for an inner product space V. Show that if w is a vector in V, then 
25 ' IM| 2 = (w,Y!} 2 4 (w,y 2 } 2 + (w,v 3 } 2 . 

Let (vj, \' 2 ,..., v H ) be an orthonormal basis for an inner product space V. Show that if w is a vector in V, then 

26. n 2 / 12 i j i2 i ,2 

IWI =(W,Y l) +(w,V 2 ) +-+(w,V H )- 


In Step 3 of the proof of Theorem 6.3.6, it was stated that “the linear independence of (uj, 112 , u M } ensures that 
^' v 3 i 0 ” Prove this statement. 



Prove that the diagonal entries of R in Formula 9 are nonzero. 


28. 


29. (For Readers Who Have Studied Calculus) 


Let the vector space p 2 have the inner product 



p(x)q(x)dx 


Apply the Gram-Schmidt process to transform the standard basis S= {1, jr, x 2 ) into an orthonormal basis. (The 
polynomials in the resulting basis are called the first three normalized Legendre polynomials.) 


30. (For Readers Who Have Studied Calculus) 

Use Theorem 6.3.1 to express the following as linear combinations of the first three normalized Legendre polynomials 
(Exercise 29). 


(a) 1 + x -| Ax 2 

(b) 2 — lx 2 

(c) 4 + 3* 


31. (For Readers Who Have Studied Calculus) 

Let p 2 have the inner product 

(P> 9} = j P<x)q(x)dx 

Apply the Gram-Schmidt process to transform the standard basis S= { \,x, x 2 ) into an orthonormal basis. 

Prove Theorem 6.3.2. 

32. 


Prove Theorem 6.3.5. 

33. 


Discussion 
Discovery 34 

(a) It follows from Theorem 6.3.6 that every plane through the origin in £ 3 must have an 
orthonormal basis with respect to the Euclidean inner product. In words, explain how 
you would go about finding an orthonormal basis for a plane if you knew its equation. 




(b) Use your method to find an orthonormal basis for the plane x \ 2y —z — 0- 


Find vectors x and y in ^2 that are orthonormal with respect to the inner product 

/'u, vj = 3a i vj 4- 2u2 V 2 but are not orthonormal with respect to the Euclidean inner product. 

If W is a line through the origin of p-' with the Euclidean inner product, and if a is a vector in 
36- p\ then Theorem 6.3.4 implies that u can be expressed uniquely as u = wq -I- w?, where wi is a 
vector in W and W 2 is a vector in W 1 • Draw a picture that illustrates this. 

Indicate whether each statement is always true or sometimes false. Justify your answer by 
37. giving a logical argument or a counterexample. 


(a) A linearly dependent set of vectors in an inner product space cannot be orthonormal. 

(b) Every finite-dimensional vector space has an orthonormal basis. 

(c) proj^-u is orthogonal to projj^ i u in any inner product space. 

(d) Every matrix with a nonzero determinant has a ^-decomposition. 

What happens if you apply the Gram-Schmidt process to a linearly dependent set of vectors? 

38. 
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6.4 

BEST APPROXIMATION; 
LEAST SQUARES 


In this section we shall show how orthogonal projections can be used to 
solve certain approximation problems. The results obtained in this section 
have a wide variety of applications in both mathematics and science. 


Orthogonal Projections Viewed as Approximations 


If P is a point in ordinary 3-space and W is a plane through the origin, then the point Q in W that is closest to P can be 
obtained by dropping a perpendicular from P to W (Figure 6.4.1a). Therefore, if we let u _ Qp, then the distance between P 
and W is given by 

||u —proiu/u|| 

In other words, among all vectors w in W, the vector w = proj^u minimizes the distance ||u — w|| (Figure 6.4.16). 



(a} Q is llic point in \¥ closest to P. {h\ || u - w |i is minimized by w = pm ju. 

Figure 6.4.1 


There is another way of thinking about this idea. View a as a fixed vector that we would like to approximate by a vector in 
W. Any such approximation w will result in an “error vector,” 

u — w 

that, unless u is in W, cannot be made equal to 0- However, by choosing 

w = profit 


we can make the length of the error vector 


u-w|| = ||u —proj r u|| 


as small as possible. Thus we can describe proj^u as the “best approximation” to u by vectors in W. The following theorem 
will make these intuitive ideas precise. 


THEOREM 6.4.1 

Best Approximation Theorem 

If W is a finite-dimensional subspace of an inner product space V, and ifu is a vector in V, then projp^u is the best 
approximation to u from W in the sense that 

||u-proj r u|| < ||n-w|| 

for every vector w in W that is different from proj^u. 


Proof For every vector w in W, we can write 









U - w = (u - proj w \i) 4- (proj - w) 


(1) 


But proj^u-w, being a difference of vectors in I/I/, is in I/I/; and u-proj^u is orthogonal to I/I/, so the 
two terms on the right side of 1 are orthogonal. Thus, by the Theorem of Pythagoras (Theorem 
6.2.4), 

||u - w || 2 = 11 u - proiw/ull 2 -I- 11 proi HPu - w || 2 

If w 5 tproj r u, then the second term in this sum will be positive, so 

||u-w|| 2 > ||u-proj^u || 2 


or, equivalently, 


||u — w|| > IKi-proju/ull 


Applications of this theorem will be given later in the text. 

Least Squares Solutions of Linear Systems 

Up to now we have been concerned primarily with consistent systems of linear equations. However, inconsistent linear 
systems are also important in physical applications. It is a common situation that some physical problem leads to a linear 
system A x = b that should be consistent on theoretical grounds but fails to be so because “measurement errors” in the entries 
of A and b perturb the system enough to cause inconsistency. In such situations one looks for a value of x that comes “as 
close as possible” to being a solution in the sense that it minimizes the value of ||A x: — b || with respect to the Euclidean inner 
product. The quantity || A x — b || can be viewed as a measure of the “error” that results from regarding x as an approximate 
solution of the linear system A x = b- If the system is consistent and x is an exact solution, then the error is zero, since 
|| Ax — b|| = ||0|| = 0. In general, the larger the value of ||A x — b||, the more poorly x serves as an approximate solution of the 
system. 


Least Squares Problem Given a linear system A x = b of m equations in n unknowns, find a vector x, if possible, that 
minimizes ||A x — b|| with respect to the Euclidean inner product on R m . Such a vector is called a least squares solution of 
A x = b- 


Remark To understand the origin of the term least squares, let e = Ax — b> which we can view as the error vector that 

results from the approximation x. If e = (sj, S 2 , ..., e m )> then a least squares solution minimizes ||e|| = (g^ -\ g^ H- \- J 

; hence it also minimizes || e || 2 = 4 . g^ j - 1 . e ^ . Hence the term least squares. 


To solve the least squares problem, let IT be the column space of A. For each « x 1 matrix x, the product is a linear 
combination of the column vectors of A. Thus, as x varies over R n , the vector varies over all possible linear combinations 
of the column vectors of A; that is, varies over the entire column space W. Geometrically, solving the least squares 
problem amounts to finding a vector x in R n such that Ax is the closest vector in W to b (Figure 6.4.2). 



= column space of/l 

Figure 6.4.2 

A least squares solution x produces the vector Ax in W closest to b. 




It follows from the Best Approximation Theorem (6.4.1) that the closest vector in W to b is the orthogonal projection of b on 
W. Thus, for a vector x to be a least squares solution of Ax = b, this vector must satisfy 

Ax = proj ff 7b 

One could attempt to find least squares solutions of A x = b by first calculating the vector projj^b and then solving 2; 
however, there is a better approach. It follows from the Projection Theorem (6.3.4) and Formula 5 of Section 6.3 that 

b — A x: = b — proj ffrb 

is orthogonal to W. But W is the column space of A, so it follows from Theorem 6.2.6 that b — Ax li es in the nullspace of a t . 
Therefore, a least squares solution of Ax = b must satisfy 

A r (b—Ax) =0 

or, equivalently, 

A r Ax = A r b ( 3 ) 

This is called the normal system associated with Ax — b, and the individual equations are called the normal equations 
associated with Ax = b- Thus the problem of finding a least squares solution of Ax = b has been reduced to the problem of 
finding an exact solution of the associated normal system. 

Note the following observations about the normal system: 


The normal system involves n equations in n unknowns (verify). 


The normal system is consistent, since it is satisfied by a least squares solution of Ax = b- 


The normal system may have infinitely many solutions, in which case all of its solutions are least squares solutions of 
Ax = b- 


From these observations and Formula 2, we have the following theorem. 


THEOREM 6.4.2 


For any linear system Ax = b> the associated normal system 

A T Ax = A T b 

is consistent, and all solutions of the normal system are least squares solutions of Ax = b. 
Moreover, if W is the column space of A, and x is any least squares solution of Ax = b, then the 
orthogonal projection of b on W is 

projfpb =Ax 


Uniqueness of Least Squares Solutions 

Before we examine some numerical examples, we shall establish conditions under which a linear system is guaranteed to 
have a unique least squares solution. We shall need the following theorem. 



THEOREM 6.4.3 


If A is an mxn matrix, then the following are equivalent. 
(a) A has linearly independent column vectors. 


(b) a T a I s invertible. 


Proof We shall prove that (a) =*. (b) and leave the proof that (b) => (a) as an exercise. 

Assume that A has linearly independent column vectors. The matrix A 7 " a has size n x «, so we can prove that this 
matrix is invertible by showing that the linear system A 7 " Ax = 0 has only the trivial solution. But if x is any solution of this 
system, then Ax is in the nullspace of a 7 " and also in the column space of A. By Theorem 6.2.6 these spaces are orthogonal 
complements, so part ( b ) of Theorem 6.2.5 implies that Ax = 0- But A has linearly independent column vectors, so x = 0 by 
Theorem 5.6.8. 

The next theorem is a direct consequence of Theorems Theorem 6.4.2 and Theorem 6.4.3. We omit the details. 


THEOREM 6.4.4 


If A is an mx.n matrix with linearly independent column vectors, then for every mx\ matrix b, the linear system Ax = b 
has a unique least squares solution. This solution is given by 

x= (A r A) _1 A r b (4) 

Moreover, if 1/1/ is the column space of A, then the orthogonal projection of b on W is 

proj^b = Ax = A(A T A) A r b (5) 


Remark Formulas 4 and 5 have various theoretical applications, but they are very inefficient for numerical calculations. 
Least squares solutions of Ax = b are typically found by using Gaussian elimination to solve the normal equations, and the 
orthogonal projection of b on the column space of A, if needed, is best obtained by computing Ax, where x is the least 
squares solution of Ax = b- The ^-decomposition of A is also used to find least squares solutions of Ax = b- 


Least Squares Solution 


Find the least squares solution of the linear system Ax = b given by 






*1“ *2=4 
3*1 4- 2*2 = 1 
— 2*1 4- 4*2 = 3 

and find the orthogonal projection of b on the column space of A. 


Solution 

Here 



1 

- r 


'4' 

A = 

3 

2 

and b = 

1 


-2 

4 


3 


Observe that A has linearly independent column vectors, so we know in advance that there is a unique least squares solution. 
We have 


A r A = 


1 3 -2 
-12 4 


1 - 1 
3 2 

-2 4 


14 -3 
-3 21 


A r b = 




" 4 " 



1 3 

-2' 

1 

3 


f 

-1 2 

4 _ 


.1°. 


so the normal system T ^ 7 ^ i n this case is 


Solving this system yields the least squares solution 


14 

-3' 

"*f 


f 

-3 

21 _ 

*2 


.1°. 


XL 

95’ 


143 

285 


From Formula 5, the orthogonal projection of b on the column space of A is 


Ax = 


1 

- f 

" 17 

95 

3 

2 

143 

-2 

4 

285 


92 

285 

439 

285 

94 

57 


Remark The language used for least squares problems is somewhat misleading. A least squares solution of Ax = b is not in 
fact a solution of = b unless Ax = b happens to be consistent; it is a solution of the related system Ax = A^b instead. 


Orthogonal Projection on a Subspace 


Find the orthogonal projection of the vector u = ( — 3, —3,8, 9) on the subspace of ^>4 spanned by the vectors 


ui = (3, 1,0, 1), 


u 2 = (1,2, 1,1), u 3 = (-1,0, 2, -1) 


Solution 




































One could solve this problem by first using the Gram-Schmidt process to convert (u^ 112 , U 3 } into an orthonormal basis and 
then applying the method used in Example 6 of Section 6.3. However, the following method is more efficient. 


The subspace W of j ? 4 spanned by up 112 , and 113 is the column space of the matrix 


A = 


1 -1 

2 0 

1 2 

1 -1 


Thus, if u is expressed as a column vector, we can find the orthogonal projection of a on Why finding a least squares 
solution of the system — u and then calculating proj^u = j4x from the least squares solution. The computations are as 
follows: The system = 11 is 

- l' 


so 


A T A = 


3 1 0 
1 2 1 
1 0 2 


0 

2 

- 1 


^ r n = 


3 10 1 

12 1 1 

10 2-1 


*1 

*2 

*3 


3 1 
1 2 
0 1 
1 1 

-3' 

-3 

8 

9 


-3 

-3 

8 

9 


- 1 
0 
2 

- 1 


11 6 -4 


6 7 
-4 0 


-3 


10 


The normal system ^^x = .d^u hi this case is 


11 

6 

-4" 

■*f 


' -3" 

6 

7 

0 

x 2 

= 

3 

-4 

0 

6 

*3 


10 


Solving this system yields 



■*r 


' - f 

X = 

*2 

= 

2 


*3 


1 


as the least squares solution of = 11 (verify), so 


projjp'U = j4x — 


3 1 
1 2 
0 1 
1 1 


- 1 

0 

2 

- 1 


- 1 
2 
1 


-2 

3 

4 
0 


or, in horizontal notation (which is consistent with the original phrasing of the problem), proj^u = ( — 2 , 3 , 4 , 0 )- 


In Section 4.2 we discussed some basic orthogonal projection operators on p' A and j ? 3 (Tables 4 and 5). The concept of an 
orthogonal projection operator can be extended to higher-dimensional Euclidean spaces as follows. 


DEFINITION 









































If W is a subspace of £ m , then the transformation p p m _> if that maps each vector x in p m into its orthogonal 

projection proj x in W is called the orthogonal projection of R m on W. 


We leave it as an exercise to show that orthogonal projections are linear operators. It follows from Formula 5 that the 
standard matrix for the orthogonal projection of R m on W is 

[P] =A(A T A)~ 1 A T (6) 

where A is constructed using any basis for W as its column vectors. 


Verifying Formula (6) 


In Table 5 of Section 4.2 we showed that the standard matrix for the orthogonal projection of p} on the xy-plane is 


[P] 


1 0 0 
0 1 0 
0 0 0 


(7) 


To see that this is consistent with Formula 6, take the unit vectors along the positive x and y axes as a basis for the xy-plane, 
so that 


A = 


1 

0 

0 


0 

1 

0 


We leave it for the reader to verify that A 7 A is the 2x2 identity matrix; thus Formula 6 simplifies to 



'1 o' 



'1 0 o' 

[P] =aa t = 

0 1 

1 0 0 

0 1 0 

= 

0 1 0 


0 0 



0 0 0 


which agrees with 7. 


♦ 


Standard Matrix for an Orthogonal Projection 


Find the standard matrix for the orthogonal projection P of p} on the line l that passes through the origin and makes an angle 
d with the positive x-axis. 


Solution 


The line / is a one-dimensional subspace of p 2 . As illustrated in Figure 6.4.3, we can take v = (cos 9, sin#) as a basis for this 
subspace, so 

cos 9 
sin#_ 

We leave it for the reader to show that a T A is the 1 x 1 identity matrix; thus Formula 6 simplifies to 


[P] =AA T 


cos Q 
sin 9 


[cos 8 


sin@] = 


cos 2 8 
sm8 cos 8 


sin$ cos 8 
sin 2 8 
























Note that this agrees with Example 6 of Section 4.3. 




Figure 6.4.3 

Summary 

Theorem 6.4.3 enables us to add yet another result to Theorem 6.2.7. 


♦ 


THEOREM 6.4.5 


Equivalent Statements 

If A is an nxn matrix, and if 7_&.R n —* Pf is multiplication by A, then the following are equivalent. 

(a) A is invertible. 


(b) = 0 has only the trivial solution. 


(c) The reduced row-echelon form of A is [ . 

(d) A is expressible as a product of elementary matrices. 


( e ) = b is consistent for every nx \ matrix b. 








(f) = b has exactly one solution for every « x 1 matrix b. 

(g) det(y4) * 0- 

(h) The range of T^is Rf. 

(i) is one-to-one. 

(j) The column vectors of A are linearly independent. 

(k) The row vectors of A are linearly independent. 

(l) The column vectors of A span R n . 

(m) The row vectors of A span Rf. 

(n) The column vectors of A form a basis for R n . 

(o) The row vectors of A form a basis for R' 1 . 

(p) A has rank n. 

(q) A has nullity 0. 

(r) The orthogonal complement of the nullspace of A is R n . 

(s) The orthogonal complement of the row space of A is {0}. 

(t) A is invertible. 

This theorem relates all of the major topics we have studied thus far. 

Exercise Set 6.4 


Click here for Just Ask! 






Find the normal system associated with the given linear system. 


(a) 


1 - 1 

2 3 

4 5 


*1 

x 2 


2 

- 1 

5 


(b) 


2 

3 

1 

1 


■1 0 
1 2 
4 5 
2 4 


r*r 


' - f 


0 

*2 

= 

1 





2 


2 . 


In each part, find det(A ; A), and apply Theorem 6.4.3 to determine whether A has linearly independent column vectors. 


(a) -13 2 

A= 2 13 
0 1 1 


(b) 



-1 3 

1 1 

0 -2 

-5 3 


Find the least squares solution of the linear system Ax = b, and find the orthogonal projection of b onto the column spac< 
3. of A. 


(a) 

1 f 


7" 

A = 

-1 1 

,b = 

0 


-1 2 


-7 

(b) 

'2 -2" 


2" 

A = 

1 1 

,b = 

- 1 


3 1 


1 


(c) 

A = 


1 

0 

- 1 


6 

2 

1 

-2 

> b = 

0 

1 

1 

0 

9 

1 

1 

- 1 


3 



'2 

0 

- f 


‘o' 

1 

-2 

2 

,b = 

6 

2 

- 1 

0 

0 

0 

1 

- 1 


6 


Find the orthogonal projection of u onto the subspace of r} spanned by the vectors vj and \' 2 - 

4. 


(a) 


u=(2, 1,3); vi = (1,1,0), v 2 = (1,2,1) 


(b) u= (1, -6, 1); vi = O 1, 2, 1), v 2 = (2, 2,4) 


Find the orthogonal projection of u onto the subspace of R 4 spanned by the vectors vp \ r 2 , and v 3 . 

5. 


(a) u= (6, 3, 9, 6); vi = (2, 1, 1, 1), v 2 = (1, 0, 1, 1), v 3 = (-2, - 1, 0, - 1) 


(b) u=( —2,0, 2,4 );vi = (1,1,3,0),v 2 = ( —2, -1, - 2, 1), v 3 = ( - 3, -1,1,3) 


Find the orthogonal projection of u = (5 ; 6, 7, 2) onto the solution space of the homogeneous linear system 

*1 I *2 "F *3 =0 

2*2 I *3 H *4=0 


Use Formula 6 and the method of Example 3 to find the standard matrix for the orthogonal projection p- R 2 

1 . 


(a) the .I'-axis 


(b) the v-axis 


Note Compare your results to Table 4 of Section 4.2. 

Use Formula 6 and the method of Example 3 to find the standard matrix for the orthogonal projection p. r-‘ 


(a) the 7lZ -plane 


(b) the yz-plane 


R 2 onto 


4 r 2 onto 


Note Compare your results to Table 5 of Section 4.2. 



Show that if w = (a, b, c) is a nonzero vector, then the standard matrix for the orthogonal projection of onto the line 
span {w} is 


P = 


2 , ,2 . 2 
a 4- b H c 


a 2 

ab 

ac 

ab 

b 2 

be 

ac 

be 

c 2 


Let IF be the plane with equation 5x — 3y A z = 0- 

10 . 

(a) Find a basis for IF. 

(b) Use Formula 6 to find the standard matrix for the orthogonal projection onto IF. 

(c) Use the matrix obtained in (b) to find the orthogonal projection of a point 70- z 0) onto 

(d) Find the distance between the point .Pg(l, — 2, 4) and the plane IF, and check your result using Theorem 3.5.2. 

Let W be the line with parametric equations 

x = 2t, y = —t, z = 4t ( — oo <t < oo) 

(a) Find a basis for W. 

(b) Use Formula 6 to find the standard matrix for the orthogonal projection onto W. 

(c) Use the matrix obtained in (b) to find the orthogonal projection of a point .Pq(x[|, y$, zq) on t° IF. 

(d) Find the distance between the point Pq(2, 1, — 3) and the line W. 

In p}, consider the line l given by the equations = t, y — t, z — t } and the line m given by the equations 
12- {x = s,y = 2s-\,z=\}- Let P be a point on /, and let Q be a point on m. Find the values of t and 5 that minimize the 
distance between the lines by minimizing the squared distance ||,P — Q\\^. 

For the linear systems in Exercise 3, verify that the error vector — b resulting from the least squares solution x is 

13. orthogonal to the column space of A. 

Prove: If A has linearly independent column vectors, and if j4x = b is consistent, then the least squares solution of j4x = b 

14. and the exact solution of = b are the same. 

Prove: If A has linearly independent column vectors, and if b is orthogonal to the column space of A, then the least 

15. squares solution of j4x = b is x = 0- 


16 . 


Let p p m _> if be the orthogonal projection of p m onto a subspace W. 


(a) Prove that [,P] 2 = [P]. 


(b) What does the result in part (a) imply about the composition p 0 p? 

(c) Show that [P] is symmetric. 

(d) Verify that the matrices in Tables 4 and 5 of Section 4.2 have the properties in parts (a) and (c). 


Let A be an m x n matrix with linearly independent row vectors. Find a standard matrix for the orthogonal projection of 

17. p n onto the row space of A. 

Hint Start with Formula 6. 


The relationship between the current I through a resistor and the voltage drop V across it is given by Ohm's Law p — IR- 

18. Successive experiments are performed in which a known current (measured in amps) is passed through a resistor of 
unknown resistance R and the voltage drop (measured in volts) is measured. This results in the (R p) data (0.1, 1), (0.2, 
2.1), (0.3, 2.9), (0.4, 4.2), (0.5, 5.1). The data is assumed to have measurement errors that prevent it from following 
Ohm's Law precisely. 


(a) Set up a 5 x 1 linear system that represents the 5 equations — RPq, .— Rp^- 

(b) Is this system consistent? 

(c) Find the least squares solution of this system and interpret your result. 


Repeat Exercise 18 under the assumption that the relationship between the current I and the voltage drop V is best 

19. modeled by an equation of the form P — IR-\- c, where c is a constant offset value. This leads to a 5 x 2 linear system. 

Use the techniques of Section 4.4 to fit a polynomial of degree 4 to the data of Exercise 18. Is there a physical 

20 . interpretation of your result? 


Discussion 

Discovery 


The following is the proof that (R) =$. (a) in Theorem 6.4.3. Justify each line by filling in the 
blank appropriately. 


Hypothesis: Suppose that A is an m x n matrix and is invertible. 


Conclusion: A has linearly independent column vectors. 





Proof: 


1. If x is a solution of Ax = 0, then A T Ax — Q._ 

2. Thus, x = 0-_ 

3. Thus, the column vectors of A are linearly independent. 


Let Abean^xj! matrix with linearly independent column vectors, and let b be an m x 1 
22. matrix. Give a formula in terms of A and for 

(a) the vector in the column space of A that is closest to b relative to the Euclidean inner 
product; 

(b) the least squares solution of Ax — b relative to the Euclidean inner product; 

(c) the error in the least squares solution of Ax = b relative to the Euclidean inner product; 

(d) the standard matrix for the orthogonal projection of R m onto the column space of A 
relative to the Euclidean inner product. 


Refer to Exercises 18-20. Contrast the techniques of polynomial interpolation and fitting a line 
23. by least squares. Give circumstances under which each is useful and appropriate. 
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CHANGE OF BASIS 


6.5 


A basis that is suitable for one problem may not be suitable for another, so it is 
a common process in the study of vector spaces to change from one basis to 
another. Because a basis is the vector space generalization of a coordinate 
system, changing bases is akin to changing coordinate axes in ft 2 and ft 3 . In 
this section we shall study problems related to change of basis. 


Coordinate Vectors 

Recall from Theorem 5.4.1 that if s= {vj, \ r 2 ,v H } is a basis for a vector space V, then each vector v in V can be expressed 
uniquely as a linear combination of the basis vectors, say 

v = jfci V! -I- k 2 V 2 4- - 4- k n v H 

The scalars k\>k 2 > ■ ■ •> k n are the coordinates of v relative to S, and the vector 

(y)s= (k l ,k 2 ,..., k») 

is the coordinate vector of v relative to S. In this section it will be convenient to list the coordinates as entries of an n x 1 matrix. 
Thus we take 



to be the coordinate vector of v relative to S. 

Change of Basis 

In applications it is common to work with more than one coordinate system, and in such cases it is usually necessary to know the 
relationships between the coordinates of a fixed point or vector in the various coordinate systems. Since a basis is the vector 
space generalization of a coordinate system, we are led to consider the following problem. 

Change-of-Bas If we change the basis for a vector space V from some old basis B to some new basis B f , how is the 

old coordinate vector [ v ] B of a vector v related to the new coordinate vector [v] 5'? 

For simplicity, we will solve this problem for two-dimensional spaces. The solution for n-dimensional spaces is similar and is left 
for the reader. Let 


5= {111,112} and B* = {u'i, u' 2 } 


be the old and new bases, respectively. We will need the coordinate vectors for the new basis vectors relative to the old basis. 
Suppose they are 



( 1 ) 


That is. 


u 1 = a uj F&U2 
1/2 = c 111 I «2 


( 2 ) 


Now let v be any vector in V, and let 



(3) 


be the new coordinate vector, so that 












v = £iu'i \-k2\\*2 


(4) 


In order to find the old coordinates of v, we must express v in terms of the old basis B. To do this, we substitute 2 into 4. This 
yields 

v = k\(a ill + & u 2) + ^2 ( c U 1 + & u 2) 


or 

Thus the old coordinate vector for v is 


v — (*1 a T ^2 c) u l + (jtl & T ^2 d)\\2 


_ k\a-\-k2C 
[V]B_ k\b + k 2 d 

which can be written as 



~a c “ 

jfcl 


“a c~ 

[v] B = 

b d_ 

k2 

or ? from (3), [v] g = 

- b d . 


This equation states that the old coordinate vector [ v ] B results when we multiply the new coordinate vector [v] on the left by 
the matrix 


P = 


a c 
b d 


The columns of this matrix are the coordinates of the new basis vectors relative to the old basis [see 1], Thus we have the 
following solution of the change-of-basis problem. 


Solution of the Cli If we change the basis for a vector space V from the old basis B = (ui, 112 ,..u H } 

to the new basis B ! = if 2 ,if n |, then the old coordinate vector [ v ] B of a vector v is related to the new coordinate vector 
[ v] r. 1 of the same vector v by the equation 


[v] b = / ? [v] B ' 


(5) 


where the columns of P are the coordinate vectors of the new basis vectors relative to the old basis; that is, the column vectors of 
P are 


[-'‘If ["'4b.["'4b 


Transition Matrices 

The matrix P is called the transition matrix from B* to B; it can be expressed in terms of its column vectors as 

^=[["'4b|["'4bH["'4b] 


Finding a Transition Matrix 


Consider the bases B = { 111 , 112 } and B 1 = {u |f 1 , 11 ^ 2 } for p}, where 

ui = 0,0); 112 =(0,1); u'i = (1, 1); u' 2 = (2,l) 














(a) Find the transition matrix from B 1 to B. 

(b) Use 5 to find [ v ] s if 

[v] s' - 


Solution (a) 

First we must find the coordinate vectors for the new basis vectors Vi and uS relative to the old basis B. By inspection, 

u'i =ui + "2 
\\ ! 2 = 2 ui + u 2 
so 

[«i]b' = 

Thus the transition matrix from B 1 to B is 


and [u 2 ]£' = 


P = 


1 2 
1 1 


Solution (b) 

Using 5 and the transition matrix in part (a) yields 

1 

2 _ 

As a check, we should be able to recover the vector v either from [ v ] s or [v] B <. We leave it for the reader to show that 
— 3V i 4- 5 u' 2 = 7 ui H 2 U 2 = v = (7, 2)• 



♦ 


A Different Viewpoint on Example 1 


Consider the vectors uj = (1, 0), 112 = (0, 1), u / 1 = fl, 1), V 2 = (2, 1). In Example 1 we found the transition matrix from the 
basis B 1 = {V 1 , V 2 } for R 2 to the basis B = {ui, u 2 } • However, we can just as well ask for the transition matrix from B to B ! ■ 
To obtain this matrix, we simply change our point of view and regard b' as the old basis and B as the new basis. As usual, the 
columns of the transition matrix will be the coordinates of the new basis vectors relative to the old basis. 


By equating corresponding components and solving the resulting linear system, the reader should be able to show that 

ui = - Vi 4 1/2 
u 2 = 2 u 1 - u 2 


so 


[«1 ] B ‘ = 


and 


[U2]fi' = 


2 

- 1 


Thus the transition matrix from B to B : is 






















Q = 


-1 2 
1 -1 


♦ 


If we multiply the transition matrix from B r to B obtained in Example 1 and the transition matrix from B to B : obtained in 
Example 2, we find 


PQ = 


'1 2 ' 

' - 1 2 ' 


'1 O ' 

_1 1 _ 

1 - 1 _ 


_° 1 _ 


= l 


which shows that Q = P 1 


The following theorem shows that this is not accidental. 


THEOREM 6.5.1 


If P is the transition matrix from a basis B r to a basis Bfor a finite-dimensional vector space V, then P is invertible, and P 1 
is the transition matrix from B to B ! - 


Let Q be the transition matrix from B to B' ■ We shall show that PQ = / and thus conclude that Q = P 1 to complete the 

proof. 


Assume that B = {uj, 112 ,u H } and suppose that 


From 5, 



^11 

C12 

c \n 

PQ = 

c 2\ 

^22 

c 2n 


c n\ 

c n2 


[i] B = ,P[x] 


and 



for all x in V. Multiplying the second equation through on the left by P and substituting the first gives 


[x] B = PQ[*]b 


for all x in V. Letting x = uj in 7 gives 


r 


'^11 

c 12 ■ 

" c 1 h 


T 


T 


’^n ’ 

0 

= 

^21 

c 22 ■ 

" c 2n 


0 

or 

0 

= 

C 21 

0 


c n\ 

c n2 " 

" c nn 


0 


0 


c h 1 


Similarly, successively substituting x = 112 , • •., u M in 7 yields 


'C12 _ 


'o' 




'o' 

r-i 

r-i ■ ■ 
r o 

= 

1 

? - " -? 

c 2n 

= 

0 

c n2 


0 


c nn 


1 


Therefore, PQ = [. 


(7) 


To summarize, if P is the transition matrix from a basis B ! to a basis B, then for every vector v, the following relationships hold: 

['’]* = £* Mb' 


( 8 ) 































(9) 


[v]b' = -P - 1 [y]b 


Exercise Set 6.5 



Click here for Just Ask! 


Find the coordinate vector for w relative to the basis S= {ui, 112 } f° r R 2 - 


(a) U1 = (1, 0),u 2 = ( 0 , 1); w= (3, -7) 


(b) m = (2, —4), u 2 = (3, 8 ); w= (1, 1) 


( c ) ui = (1, 1)» 112 = (0, 2); w= (a, i) 


Find the coordinate vector for v relative to S = {vi, V 2 , v 3 j • 


(a) v = (2, -1,3);vi(1,0,0),v 2 (2,2,0),v 3 (3,3,3) 


(b) v = (5, — 12, 3); vi(l, 2, 3). v 2 ( — 4, 5, 6 ), v 3 (7, -8,9) 


Find the coordinate vector for p relative to s= (pi, P2> P 3 } • 


(a) p = 4 _ 3 * + x 2 ; P1 = 1 , p 2 = x, p, = x 2 

(b) p = 2 — x + x 2 ', p 1 = 1 4 - x> p 2 = 1 + x 2 > p 3 = x 4- x 2 


4 . 


Find the coordinate vector for A relative to s= {A\, A 2 , A-$, A 4 } • 


A = 


2 O' 
_ -1 3_ 

II 

'-1 f 
0 0_ 

II 

C-l 

'1 r 
_0 0_ 

II 

I"! 

0 0 

0 l— 1 

1_1 

II 

’=d _ 

0 l— 1 

0 0 

1_1 


Consider the coordinate vectors 



r 

L 


r 

L 


oc 

1 

II 

6 

II 

3 

II 

7 

- 1 

4 

0 

4 

6 






3 




















(a) Find w if S is the basis in Exercise 2(a). 

(b) (b) Find q if S is the basis in Exercise 3(a). 

(c) (c) Find B if S is the basis in Exercise 4. 


Consider the bases B = ( 111 , 112 ) and B 1 = (vi,V 2 ) for where 



V 


' 0 ‘ 


" 2 " 


' - 3" 

111 = 

0 

. u 2 = 

1 

. vi = 

1 

, and \'2 

4 


(a) Find the transition matrix from B 1 to B. 

(b) Find the transition matrix from B to B*■ 


(c) Compute the coordinate vector [ w ] B , where 

3' 

W= [- 5 J 

and use 9 to compute [w] B \ 

(d) Check your work by computing [w] directly. 


Repeat the directions of Exercise 6 with the same vector w but with 



"2" 


4' 


V 


■ - r 

ui = 

2 

. "2 = 

- 1 

■ vi = 

3 

. v 2 = 

- 1 


Consider the bases B = {uj, 112,113} and B 1 = (vj, V2, V3} for p}, where 



■ - 3 ' 


' -3' 


r 


' - 6' 


'-2 

ui = 

0 

-3 

. u 2 = 

2 

-1 

. u 3 = 

6 

-1 

. vi - 

— 6 

0 

. va = 

-6 

4 


(a) Find the transition matrix from B to B 1 - 


(b) Compute the coordinate vector [ w ] s , where 


w = 


-5 

8 

-5 


-2 

v 3 = -3 
7 


and use 9 to compute [w] B \ 



(c) Check your work by computing [w] directly. 


Repeat the directions of Exercise 8 with the same vector w, but with 

9 [ 2 ] r 2 ] r 11 [ 3 ] ii r-r 

Ul = 1 , 112 = — 1 , 113 = 2 , Vi = 1 , V 2 = 1 , V 3 = 0 

lj [ lj [lj [ — 5 J [- 3 J [ 2 

Consider the bases B = (m, \> 2 ) and B 1 = ((p, < 12 ) f° r P\i where 

pi = 6 4- 3x, P 2 = 10 4 2x, qi = 2, q 2 = 3 h 2* 

(a) Find the transition matrix from B : to B. 

(b) Find the transition matrix from B to B ! ■ 

(c) Compute the coordinate vector [p] s , where p = — 4 I x, and use 9 to compute [p] B <. 

(d) Check your work by computing [p] B < directly. 

Let V be the space spanned by f j = s inx and f 2 = cos x■ 

Us 

(a) Show that = 2 sinx + cos x and g 2 = 3 cos x form a basis for V. 

(b) Find the transition matrix from B'' = {gi,g2} to B= {f 1 , f 2 } • 

(c) Find the transition matrix from B to B 1 - 

(d) Compute the coordinate vector [h] s , where h = 2 sin* — 5 cos x, and use 9 to obtain [h] s \ 

(e) Check your work by computing [li] s > directly. 

If P is the transition matrix from a basis B f to a basis B, and Q is the transition matrix from B to a basis C, what is the 
12. transition matrix from B r to C? What is the transition matrix from C to p; ! - 


13 . 


Refer to Section 4.4. 



(a) Identify the bases for p 3 used for interpolation in the standard form (found by using the Vandermonde system), the 
Newton form, and the Lagrange form, assuming = _ f, x j = Q, and ^ = 1 - 


(b) What is the transition matrix from the Newton form basis to the standard basis? 


To write the coordinate vector for a vector, it is necessary to specify an order for the vectors in the basis. If P is the transition 
14. matrix from a basis B 1 to a basis B, what is the effect on P if we reverse the order of vectors in B from vj, ..., v„ to v„, .. 
vi? What is the effect on P if we reverse the order of vectors in both B' and />’? 


Discussion 

Consider the matrix 


P = 


1 1 


0 

2 

1 


(a) P is the transition matrix from what basis B to the standard basis S= {v\, e?, £ 3 } f° r j? 3 ? 

(b) P is the transition matrix from the standard basis S= e?, £ 3 } to what basis B for 


The matrix 

16. 


P = 


0 1 


0 

2 

1 


is the transition matrix from what basis B to the basis {(1, 1, 1), (1, 1, 0), (1, 0, 0)} for p}l 

If [ w ] £ = w holds for all vectors w in R } \ what can you say about the basis B1 

17. 

Indicate whether each statement is always true or sometimes false. Justify your answer by giving a 

18. logical argument or a counterexample. 


(a) Given two bases for the same inner product space, there is always a transition matrix from 
one basis to the other basis. 


(b) The transition matrix from B to B is always the identify matrix. 


(c) Any invertible n x n matrix is the transition matrix for some pair of bases for R n . 
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C C In this section we shall develop properties of square matrices with ortho normal 

column vectors. Such matrices arise in many contexts, including problems 
ORTHOGONAL MATRICES involving a change from one orthonormal basis to another. 


Matrices whose inverses can be obtained by transposition are sufficiently important that there is some terminology associated with 
them. 


DEFINITION 


A square matrix A with the property 


is said to be an orthogonal matrix. 


A~ X =A 7 


It follows from this definition that a square matrix A is orthogonal if and only if 

aa t =a t a=i 

In fact, it follows from Theorem 1.6.3 that a square matrix A is orthogonal if either _ / or jp£ — /. 


EXAMPL A 3 x 3 Orthogonal Matrix 


The matrix 


is orthogonal, since 


12 6 
7 7 7 

J 3 2 

7 7 7 

2 6 _ 3 

7 7 7 


" 3 6 2 " 

3 2 6" 

111 

7 7 7 

2 3 6 

6 3 2 

111 

7 7 7 

6 2 3 

2 6 3 

111 

7 7 7 


1 0 0 
0 1 0 
0 0 1 


♦ 


EXAMPLE 2 A Rotation Matrix Is Orthogonal 


Recall from Table 6 of Section 4.2 that the standard matrix for the counterclockwise rotation of R 2 through an angle 0 is 




















This matrix is orthogonal for all choices of ft since 


a t a= 


cos 9 

sin# 

cos 9 

— sin 9 


'i 

o' 

— sin 9 

cos# 

sin# 

cos Q 


_0 

1_ 


In fact, it is a simple matter to check that all of the “reflection matrices” in Tables 2 and 3 and all of the “rotation matrices” in 
Tables 6 and 7 of Section 4.2 are orthogonal matrices. 


Observe that for the orthogonal matrices in Examples Example 1 and Example 2, both the row vectors and the column vectors form 
orthonormal sets with respect to the Euclidean inner product (verify). This is not accidental; it is a consequence of the following 
theorem. 


THEOREM 6.6.1 

The following are equivalent for an n~xn matrix A. 

(a) A is orthogonal. 

(b) The row vectors of A form an orthonormal set in R n with the Euclidean inner product. 

(c) The column vectors of A form an orthonormal set in R n with the Euclidean inner product. 


Proof We shall prove the equivalence of ( a ) and ( b ) and leave the equivalence of (a) and (c) as an exercise. 


The entry in the ith row and jth column of the matrix product AA T is the dot product of the ith row vector of A and the jth 
column vector of A^- But except for a difference in notation, the jth column vector of a T is the jth row vector of A. Thus, if the row 
vectors of A are ^ r n , then the matrix product AA T can be expressed as 


Thus AA T = 1 if and on ly if 



l’l ■ l’l 

l'l-r2 - 

l’l i’ 

aa t = 

i"2 ■ l’l 

i’2-i’2 

1’2 1’ 


I’m • l’l 

I’m-1’2 - 

I’m -1’ 

l'l - l’l 

= r 2 - r 2 

= — = r H 

' I’m = 


and 


r 2 - i'j = 0 when i ^ j 

which are true if and only if {r 1? r 3 ,r H } is an orthonormal set in R n . 


Remark In light of Theorem 6.6.1, it would seem more appropriate to call orthogonal matrices orthonormal matrices. However, 
we will not do so in deference to historical tradition. 


The following theorem lists some additional fundamental properties of orthogonal matrices. The proofs are all straightforward and 
are left for the reader. 













THEOREM 6.6.2 


(a) The inverse of an orthogonal matrix is orthogonal. 


(b) A product of orthogonal matrices is orthogonal. 


(c) If A is orthogonal, then det(A') = 1 or det(^) = - 1- 


det< jA) = ± 1 for an Orthogonal Matrix A 


The matrix 


A = 


1/^2 1/^2 
1/^2 1/^2 


is orthogonal since its row(and column) vectors form orthonormal sets in We leave it for the reader to check that dr.t(A) = 1- 
Interchanging the rows produces an orthogonal matrix for which det(j4) = — 1- 

♦ 


Orthogonal Matrices as Linear Operators 

We observed in Example 2 that the standard matrices for the basic reflection and rotation operators on pf and pf are orthogonal. 
The next theorem will help explain why this is so. 

THEOREM 6.6.3 

If A is an n x n matrix, then the following are equivalent. 

(a) A is orthogonal 

(b) U4x|| = ||x|| for all x in R n . 

(c) j4x - j4y = x - y for all x andy in R }1 . 


Proof We shall prove the sequence of implications ( a ) =$* (b) => (c) => (a). 

(a) => (h) Assume that A is orthogonal, so that ^4 = /. Then, from Formula 8 of Section 4.1, 

||Ax|| = (^x-Ax ) 1/2 = (x-A T Ax ) i/2 = (x-x ) 1/2 = ||x| 








Assume that ||j4x|| = ||x|| for all x in R n . From Theorem 4.1.6 we have 

^■^y = i||^ + ^y|| 2 -I|| J 4 X - J 4y|| 2 = I||^(x + y)|| 2 -ip(x-y)|| 2 
= ^l|x + y|| 2 -^ll*-y|| 2 = x-y 

(c)^ Assume that j4x - j4y = x • y for all x andy in R }1 . Then, from Formula 8 of Section 4.1, we have 

i - y = x - A 1 j4v 

which can be rewritten as 

x - (j4 ^Ay — y) = 0 or x - (j4 ^A — I)y = 0 
Since this holds for all x in R n , it holds in particular if 

X - (A T A -1) y, so (A T A -1 )y - (A T A - /)y = 0 

from which we can conclude that 

(A r A-I) y = 0 (2 ; 

(why?). Thus 2 is a homogeneous system of linear equations that is satisfied by every y in R n . But this implies that the coefficient 
matrix must be zero (why?), so A T A — I and, consequently, A is orthogonal. 

If T: R n -* R n is multiplication by an orthogonal matrix A, then T is called an orthogonal operator on R n . It follows from parts 

(a) and ( b ) of the preceding theorem that the orthogonal operators on R n are precisely those operators that leave the lengths of all 
vectors unchanged. Since reflections and rotations of pi and g-' have this property, this explains our observation in Example 2 that 
the standard matrices for the basic reflections and rotations of R 2 and R 3 are orthogonal. 

Change of Orthonormal Basis 

The following theorem shows that in an inner product space, the transition matrix from one orthonormal basis to another is 
orthogonal. 


THEOREM 6.6.4 


If P is the transition matrix from one orthonormal basis to another orthonormal basis for an inner product space , then P is an 
orthogonal matrix; that is, 

p~ l =p T 


Assume that V is an n-dimensional inner product space and that P is the transition matrix from an orthonormal basis to an 
orthonormal basis B. To prove that P is orthogonal, we shall use Theorem 6.6.3 and show that \\Px\\ = ||x|| for every vector x in R n . 

Recall from Theorem 63.2a that for any orthonormal basis for V, the norm of any vector u in V is the same as the norm of its 
coordinate vector in R n with respect to the Euclidean inner product. Thus for any vector u in V, we have 

Nl = ll[ u ]£-'ll = \\[k]b\\ 

or 

IN = \\[k]b'\\ = (3) 

where the first norm is with respect to the inner product on V and the second and third are with respect to the Euclidean inner 
product on R n . 



Now let x be any vector in and let u be the vector in V whose coordinate vector with respect to the basis £ ! is x; that is, 
[u] ■■ = x. Thus, from 3, 

Hull = ||x|| = ||Px|| 


which proves that P is orthogonal. 


EXAMPLE 4 Application to Rotation of Axes in 2-Space 


f f 

In many problems a rectangular ^-coordinate system is given, and a new x'y -coordinate system is obtained by rotating the xy 
-system counterclockwise about the origin through an angle Q . When this is done, each point Q in the plane has two sets of 
coordinates: coordinates (*, y) relative to the xy-system and coordinates (x\ y ! ) relative to the x'y ; -system (Figure 6.6.1a). 


By introducing unit vectors uj and 112 along the positive x- and y-axes and unit u ' and u ' along the positive and y '-axes, we can 
regard this rotation as a change from an old basis £ = (uj, 112 } to a new basis B ! = (uj, 112 } (Figure 6.6.1 /;). Thus, the new 
coordinates {x ! , y ! ) and the old coordinates (%, y ) of a point Q will be related by 


y 


= p~ 


(4) 


where P is the transition from £ ! to B. To find P we must determine the coordinate matrices of the new basis vectors u ' and 
relative to the old basis. As indicated in Figure 6.6.1c, the components of u ' in the old basis are f) and sm 0 , so 

cos 8 


K] s = 


sin^ 


Similarly, from Figure 6.6.1 d, we see that the components of in the old basis are cos(0 + tc / 2) = — sm& and 
sm(9 + 7T / 2) = cos ft so 


Mb = 


— sin^ 
cos 0 




ic) 



Figure 6.6.1 

Thus the transition matrix from B f to B is 


P = 


cos 9 
sin@ 


- sinfl 
cos 9 


Observe that P is an orthogonal matrix, as expected, since B and B f are orthonormal bases. Thus 
























so 4 yields 


p~ l =p T 


cos # sin# 
— sin# cos# 


1 

“ t - 

K 

1 _ 

l 

-1 

/ 

l 

1- 


cos# sin# 
— sin# cos# 


or, equivalently, 

x f = x cos# + 7 sin# 
y f = —x sin #-\-y cos# 

For example, if the axes are rotated # = ^ / 4, then since 

sin ~ = cos -7 = —1= 

Equation 5 becomes 

1 1 

ft x 

_j_ _l y 

{2 p> 

Thus, if the old coordinates of a point Q are y ) = (2, — 1), then 




1 1 


V 


{2 {2 

2 

_/ 


1 1 

p. {2 

_ - 1 _ 


1 

3 

/2 


so the new coordinates of Q are (x f , y ! ) = (1 / p 2 , — 3 / \[ 2 ). 


( 5 ) 


♦ 


Remark Observe that the coefficient matrix in 5 is the same as the standard matrix for the linear operator that rotates the vectors of 
y through the angle — 0 (Table 6 of Section 4 . 2 ). This is to be expected since rotating the coordinate axes through the angle 0 with 
the vectors of P 2 kept fixed has the same effect as rotating the vectors through the angle — 0 with the axes kept fixed. 


Application to Rotation of Axes in 3-Space 


Suppose that a rectangular xyz-coordinate system is rotated around its z-axis counterclockwise (looking down the positive z-axis) 
through an angle 0 (Figure 6 . 6 . 2 ). If we introduce unit vectors uj, 113, and 113 along the positive x-, y-, and z-axes and unit vectors u ', 
and u 3 along the positive ^ ,r , y'\ and z f axes, we can regard the rotation as a change from the old basis B = (m, 112,113} to the 
new basis B' ( = (iq, 113} . In light of Example 4 , it should be evident that 



COS # 


— sin# 

[^] B = 

sin# 

and [u^ ] B - 

cos # 


0 


0 






























■ i\ 



Figure 6.6.2 


Moreover, since extends 1 unit up the positive z ,r -axis, 


[^] B = 


Thus the transition matrix from B f to B is 


P = 


and the transition matrix from B to B' f is 


cos# —sin# 0 
sin# cos# 0 
0 0 1 

cos# sin# 0 
P 1 = — sin# cos# 0 

0 0 1 

(verify). Thus the new coordinates z ,f ) of a point Q can be computed from its old coordinates by 


cos# sin# 0 
— sin# cos# 0 
0 0 1 


♦ 


Exercise Set 6.6 



Click here for Just Ask! 


1 . 

(a) Show that the matrix 



4 

0 

3 

5 

5 

9 

4 

12 

25 

5 

25 

12 

3 

16 

25 

5 

25 


is orthogonal in three ways: by calculating a t A, by using part ( b ) of Theorem 6.6.1, and by 
using part (c) of Theorem 6.6.1. 


(b) Find the inverse of the matrix A in part (a). 





















2 . 


(a) Show that the matrix 


1 2 2 

3 3 3 

A= 1 _2 1 

A 3 3 3 

_ 2 _1 2 

3 3 3 

is orthogonal. 


(b) Let 7 : 7 3 _> 7 -' be multiplication by the matrix A in part (a). Find T(x') for the vector x = ( — 2, 3, 5)- Using the 

Euclidean inner product on verify that ||T(i) || = ||x||- 


Determine which of the following matrices are orthogonal. For those that are orthogonal, find the inverse. 

3. 


(a) 


1 0 
0 1 


(b) 


1 / {2 
1 / {l 


- 1/^2 

1 / {2 


(c) 


0 1 1/^2 
1 0 0 
0 0 \i{l 


(d) 


- 1/^2 1/^6 1/^3 

0 — 2/^6 1/^3 

1/^2 1/^6 1/^3 


(e) 


1 

2 

1 

2 

1 

2 

1 

2 


1 1 

2 2 

5 1 

6 6 

1 1 

6 6 

1 _ 5 

6 6 


1 

2 

1 

6 

5 

6 

1 

6 


(f) 


10 0 0 
0 1/^3 - 1/2 0 

0 1/^3 0 1 

0 1/^3 1/2 0 




4 . 


(a) Show that if A is orthogonal, then j T is orthogonal. 


(b) What is the normal system for = b when A is orthogonal? 


Verify that the reflection matrices in Tables 2 and 3 of Section 4.2 are orthogonal. 

5. 

Let a rectangular x'y ‘''-coordinate system be obtained by rotating a rectangular ^-coordinate system counterclockwise through 
the angle d = 3~/4- 

(a) Find the A -coordinates of the point whose ^-coordinates are (-2, 6). 

(b) Find the ?;y-coord mates of the point whose x^ ^coordinates are (5, 2). 


Repeat Exercise 6 with B = -k ! 3- 

7. 

Let a rectangular xV^-coordinate system be obtained by rotating a rectangular ^y^-coordinate system counterclockwise about 

O 

the z-axis (looking down the z-axis) through the angle Q = ^ / 4- 

(a) Find the x'VV-coordinates of the point whose xyz-coordi nates are (-1,2, 5). 

(b) Find the xyz-coordmates of the point whose x>V-coordinates are ( 1, 6, -3). 


Repeat Exercise 8 for a rotation of $ = ^ / 3 counterclockwise about the y-axis (looking along the positive y-axis toward the 
9* origin). 

Repeat Exercise 8 for a rotation of 0 = 3^ j 4 counterclockwise about the x-axis (looking along the positive x-axis toward the 
10. origin). 


11 . 


(a) A rectangular x^V-coordinate system is obtained by rotating an xyz-coordinate system counterclockwise about the 
y-axis through an angle & (looking along the positive y-axis toward the origin). Find a matrix A such that 


V 


" x~ 

/ 

ii 

ha. 

y 



z 

z 


L J 


where (x,y,z) and {x ! ,y,z ! ) are the coordinates of the same point in the xyz- and x>V-system 
respectively. 




(b) Repeat part (a) for a rotation about the x-axis. 


A rectangular x”y"z"- coordinate system is obtained by first rotating a rectangular xyz-coordinate system 60° 

*"■ counterclockwise about the z-axis (looking down the positive z-axis) to obtain an x'VV -coord in ate system, and then rotating 
the x> V-coordinate system 45° counterclockwise about the y'-axis (looking along the positive y'-axis toward the origin). Find 
a matrix A such that 




~x~ 

/' 

= A 

y 



z 

z 


L J 


where (x, y , z) and (x ", y !! , z”) are the xyz- and x”y !! z' : - coordinates of the same point. 


What conditions must a and b satisfy for the matrix 
a f b b — a 
a — b b + a 

to be orthogonal? 


Prove that a 2 x 2 orthogonal matrix A has one of two possible forms: 



cos 9 

— sin 9 

or A = 

cos 9 

sin# 

A = 

sin 9 

cos 9 

sin 9 

— cos 9 



where 0 < 9 < 2 jt. 


Hint Start with a general 2x2 matrix A = (a^), and use the fact that the column vectors form an orthonormal set in R 2 . 


15. 

(a) Use the result in Exercise 14 to prove that multiplication by a 2 x 2 orthogonal matrix is either a rotation or a rotation 
followed by a reflection about the x-axis. 

(b) Show that multiplication by A is a rotation if det(A) = 1 and that a rotation followed by a reflection if det(A) = — 1- 


Use the result in Exercise 15 to determine whether multiplication by A is a rotation or a rotation followed by a reflection about 
16. the x-axis. Find the angle of rotation in either case. 


(a) 


- 1/^2 

- 1/^2 


1 / ^2 

- 1/^2 


(b) 


A = 


- 1/2 ^ 3/2 
^ 3/2 1/2 


17 . 


The result in Exercise 15 has an analog for 3 x 3 orthogonal matrices: It can be proved that multiplication by a 3 x 3 orthogor 
matrix A is a rotation about some axis if det(A) = 1 and is a rotation about some axis followed by a reflection about some 
coordinate plane if det(A) = — 1- Determine whether multiplication by A is a rotation or a rotation followed by a reflection. 



(a) 


3 2 6 

7 7 7 

_ 6 3 2 

7 7 7 

2 6 _ 3 

7 7 7 


(b) 



1 6 

7 7 

6 2 

7 7 

2 _ 3 

7 7 


Use the fact stated in Exercise 17 and part (b) of Theorem 6.6.2 to show that a composition of rotations can always be 
18. accomplished by a single rotation about some appropriate axis. 


Prove the equivalence of statements (a) and (c) in Theorem 6.6.1. 

19. 


Discussion 

Discovery 


A linear operator on p} is called rigid if it does not change the lengths of vectors, and it is called 
angle preserving if it does not change the angle between nonzero vectors. 


(a) Name two different types of linear operators that are rigid. 


(b) Name two different types of linear operators that are angle preserving. 

(c) Are there any linear operators on p 2 that are rigid and not angle preserving? Angle preserving 
and not rigid? Justify your answer. 


Referring to Exercise 20, what can you say about det(A) if A is the standard matrix for a rigid linear 
21* operator on p^l 


22 . 


Find a, b, and c such that the matrix 


a 


A = 


b 


1/^2 - 1/^2 
1/^6 1/^6 
1/^3 1/^3 


is orthogonal. Are the values of a, b, and c unique? Explain. 
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Chapter 6 


Supplementary Exercises 

Let pA have the Euclidean inner product. 


(a) Find a vector in that is orthogonal to uj = (1, 0, 0, 0) and 114 = ( 0 , 0, 0, 1) and makes equal angles with 
u 2 = ( 0 , 1 , 0 , 0 )and „ 3 = ( 0 , 0 , 1 , 0 ). 

(b) Find a vector x = (x 1 , * 2 , * 3 , * 4 ) °f length 1 that is orthogonal to uj and 114 above and such that the cosine of the 
angle between x and 112 is twice the cosine of the angle between x and 113 . 


2 . 


Show that if x is a nonzero column vector in PJ\ then the n x n matrix 

A = l n ~—^jxx T 


is both orthogonal and symmetric. 


3. 


Let Ax — 0 he a system of m equations in n unknowns. Show that 

'*1 
*2 

x — 


is a solution of the system if and only if the vector x — (xq, x 2 ,..., x H ) is orthogonal to every row vector of A in the 
Euclidean inner product on R n . 


Use the Cauchy-Schwarz inequality to show that if a\, aj, • ••, a n are positive real numbers, then 

( fl l +fl2 + ". + flH )( J -4 ± + .„ + ±.\> n 2 


Show that if x and y are vectors in an inner product space and c is any scalar, then 
||cx h y|| 2 =c 2 ||x || 2 + 2 c(x, y) + ||y|| 2 


Let p-' have the Euclidean inner product. Find two vectors of length 1 that are orthogonal to all three of the vectors 

6 - ui = ( 1 , 1 , — 1 )» u 2 = ( — 2 , -l, 2 )and „ 3 = (_l, 0 , !)• 


Find a weighted Euclidean inner product on R n such that the vectors 






VI = (1,0, 0,..... 0) 
v 2 = (0, j/2,0,..., 0 ) 
v 3 = (0, 0, ^3 ,..., 0) 

v h = (0, o. 0,..., fit) 

form an orthonormal set. 

Is there a weighted Euclidean inner product on p2 for which the vectors (1,2) and (3,-1) form an orthonormal set? 
Justify your answer. 


Prove: If Q is an orthogonal matrix, then each entry of Q is the same as its cofactor if det(0 = 1 and is the negative of its 
9 * cofactor if <Jet(0 = - 1- 


If u and v are vectors in an inner product space V, then u, v, and u _ v can be regarded as sides of a “triangle” in V (see 
10. the accompanying figure). Prove that the law of cosines holds for any such triangle; that is, 

11 u — v 11 2 = ||u|| 2 + ||v|| 2 — 2||u|||| v||cos 9 , where 0 is the angle between u and v. 



Figure Ex-10 


11 . 

(a) In p} the vectors (k, 0, 0), (0, k, 0), and (0, 0, k ) form the edges of a cube with diagonal k, k) (Figure 3.3.4). 
Similarly, in R n the vectors 


(£,0,0.0), (0, £, 0.0). (0,0,0. k) 

can be regarded as edges of a "cube" with diagonal (k,k,k,...,k). Show that each of the 
above edges makes an angle of 9 with the diagonal, where cos 9 — 1 / 

(b) (For Readers Who Have Studied Calculus). What happens to the angle ft in part (a) as the dimension of R n 
approaches -|- oc. ? 


Let u and v be vectors in an inner product space. 

12 . 


(a) Prove that ||u|| = ||v|| if and only if u | v and u _ v are orthogonal. 


(b) Give a geometric interpretation of this result in p2 with the Euclidean inner product. 


Let u be a vector in an inner product space V, and let ( Vl , V2 ,..., v H ) be an orthonormal basis for V. Show that if a,- is 
13* the angle between u and v 3 , then 

2 . 2 i . 2 i 

cos Q’i 4- cos Q '2 -I-+ cos a n = 1 



14 . 


Prove: If (u, vj j and (u, vj, ^ are two inner products on a vector space V, then the quantity (u, vj = (u, v }l 1 ( U ’ V >2 
also an inner product. 


Show that the inner product on generated by any orthogonal matrix is the Euclidean inner product. 

15. 


Prove part (c) of Theorem 6.2.5. 

16. 
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Chapter 6 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Section 6.1 

Tl. (Weighted Euclidean Inner Products) See if you can program your utility so that it produces the value of a weighted 
Euclidean inner product when the user enters n, the weights, and the vectors. Check your work by having the program do 
some specific computations. 


T2. (Inner Product on M 22 ) $ ee if y° u can program your utility to produce the inner product in Example 7 when the user enters 
the matrices U and V. Check your work by having the program do some specific computations. 


T3. (Inner Product on C\a, &"|) If you are using a CAS or a technology utility that can do numerical integration, see if you can 
program the utility to compute the inner product given in Example 9 when the user enters a, b, and the functions f (x) and 
g(x)- Check your work by having the program do some specific calculations. 


Section 6.3 

Tl. (Normalizing a Vector) See if you can create a program that will normalize a nonzero vector v in R n when the user enters v. 


T2. (Gram-Schmidt Process) Read your documentation on performing the Gram-Schmidt process, and then use your utility to 
perform the computations in Example 7. 


T3. (^-decomposition) Read your documentation on performing the Gram-Schmidt process, and then use your utility to 
perform the computations in Example 8. 


Section 6.4 

Tl. (Least Squares) Read your documentation on finding least squares solutions of linear systems, and then use your utility to 
find the least squares solution of the system in Example 1. 


T2. 


(Orthogonal Projection onto a Subspace) Use the least squares capability of your technology utility to find the least 



squares solution x of the normal system in Example 2, and then complete the computations in the example by computing j4x- 
If you are successful, then see if you can create a program that will produce the orthogonal projection of a vector u in onto 
a subspace W when the user enters u and a set of vectors that spans W. 

Suggestion As the first step, have the program create the matrix A that has the spanning vectors as columns. 

Check your work by having your program find the orthogonal projection in Example 2. 

Section 6.5 

Tl. 

(a) Confirm that B\ = (iq, u 2 ,113,114,115} and Bj = {\q, v 2 , V3, V4, \q} are bases for rr’, and find both transition 
matrices. 


«1 = (3, 1. 3, 2, 6) 
u 2 = (4, 5, 7, 2, 4) 
113 = (3, 2, 1, 5, 4) 
114 = (2, 9, 1,4, 4) 
u 5 = (3, 3, 6 , 6 , 7) 


vi = (2,6, 3, 4, 2) 
v 2 = (3, 1,5, 8,3) 
v 3 = (5, 1, 2, 6,7) 
v 4 = (8,4, 3, 2, 6 ) 
v 5 = (5,5,6, 3,4) 


(b) Find the coordinate vectors with respect to B\ and Bj °f w = (1, 1, 1, 1, 1 ). 
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7 


CHAPTER 


Eigenvalues, Eigenvectors 


INTRODUCTION If A is an n x n matrix and xis a vector in , then Ax is also a vector in fl n , but usually there is no 
simple geometric relationship between xand Ax- However, in the special case where xis a nonzero vector and Ax is a scalar 
multiple of x, a simple geometric relationship occurs. For example, if A is a 2 x 2 matrix, and if x is a nonzero vector such that 
Ax is a scalar multiple of x, say Ax — Ax, then each vector on the line through the origin determined by x gets mapped back 
onto the same line under multiplication by A. 

Nonzero vectors that get mapped into scalar multiples of themselves under a linear operator arise naturally in the study of 
vibrations, genetics, population dynamics, quantum mechanics, and economics, as well as in geometry. In this chapter we will 
study such vectors and their applications. 
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7.1 

EIGENVALUES AND 
EIGENVECTORS 


In Section 2.3 we introduced the concepts of eigenvalue and eigenvector. In 
this section we will study those ideas in more detail to set the stage for 
applications of them in later sections. 


Review 

We begin with a review of some concepts that were mentioned in Sections 2.3 and 4.3. 


DEFINITION 


If A is an n x « matrix, then a nonzero vector x in is called an eigenvector of A if Ax is a scalar multiple of x ; that is, if 

Ax = Ax 

for some scalar A- The scalar A is called an eigenvalue of A, and x is said to be an eigenvector of A corresponding to A- 


In and g-', multiplication by A maps each eigenvector x of A (if any) onto the same line through the origin as x. Depending 
on the sign and the magnitude of the eigenvalue A corresponding to x, the linear operator Ax = Ax compresses or stretches x by a 
factor of x, with a reversal of direction in the case where \ is negative (Figure 7.1.1). 



hi) 0 < A £ I 



(b) A > ! 



(f) l<A£(> 



(d) A $ -! 


Figure 7.1.1 


Eigenvector of a 2 x 2 Matrix 


The vector x = 


1 

2 


is an eigenvector of 



corresponding to the eigenvalue A = 3, since 


Ax = 


'3 O' 

'f 


'3' 

_8 -1_ 

_2_ 


_6_ 


= 3x 


♦ 


To find the eigenvalues of an n x n matrix A, we rewrite Ax = Ax as 

Ax = A/x 




















or, equivalently, 


(XI ~ A)x = 0 


( 1 ) 


For A to be an eigenvalue, there must be a nonzero solution of this equation. By Theorem 6.4.5, Equation 1 has a nonzero 
solution if and only if 

det (XI - A) = 0 

This is called the characteristic equation of A; the scalars satisfying this equation are the eigenvalues of A. When expanded, the 
determinant det(A7 — A) is always a polynomial p in \, called the characteristic polynomial of A. 

It can be shown (Exercise 15) that if A is an n x n matrix, then the characteristic polynomial of A has degree n and the coefficient 
of A M is 1; that is, the characteristic polynomial p (A) of an n x n matrix has the form 

p(X) = det(A/-A) =A m H-ciA m_1 + - + c» 

It follows from the Fundamental Theorem of Algebra that the characteristic equation 

A H AciA h_1 H- hc H = 0 

has at most n distinct solutions, so an n x n matrix has at most n distinct eigenvalues. 

The reader may wish to review Example 6 of Section 2.3, where we found the eigenvalues of a 2 x 2 matrix by solving the 
characteristic equation. The following example involves a 3 x 3 matrix. 


Eigenvalues of a 3 x 3 Matrix 


Find the eigenvalues of 



1 0 
0 1 
-17 8 


Solution 


The characteristic polynomial of A is 


det (XI — A) = det 



- 1 
A 
17 


0 

- 1 

A — 8 


= A 3 -8A 2 + 17A-4 


The eigenvalues of A must therefore satisfy the cubic equation 


A 3 — 8A 2 4- 17A — 4 = 0 


( 2 ) 


To solve this equation, we shall begin by searching for integer solutions. This task can be greatly simplified by exploiting the 
fact that all integer solutions (if there are any) to a polynomial equation with integer coefficients 

A” + c i A” -1 + — + c H = 0 

must be divisors of the constant term, c H . Thus, the only possible integer solutions of 2 are the divisors of-4, that is, ±1, ±2, ±4. 
Successively substituting these values in 2 shows that A = 4 is an integer solution. As a consequence, A — 4 must be a factor of 
the left side of 2. Dividing A — 4 into A 3 — 8A 2 + 17A — 4 shows that 2 can be rewritten as 

(A —4) (A 2 — 4A+ 1) = 0 

Thus the remaining solutions of 2 satisfy the quadratic equation 








A — 4A + 1 = 0 

which can be solved by the quadratic formula. Thus the eigenvalues of A are 

A = 4, A = 2 4- ^3, and X = 2- ] [3 


Remark In practical problems, the matrix A is usually so large that computing the characteristic equation is not practical. As a 
result, other methods are used to obtain eigenvalues. 


Eigenvalues of an Upper Triangular Matrix 


Find the eigenvalues of the upper triangular matrix 


A = 


a ll 

«12 

a l3 

a 14 

0 

"22 

a 23 

a 24 

0 

0 

a 33 

a 34 

0 

0 

0 

a 44 


Solution 

Recalling that the determinant of a triangular matrix is the product of the entries on the main diagonal (Theorem 2.1.3), we 
obtain 


det(A/ — A) = det 


-a 11 

“«12 

-a\3 

-a M 

0 

A-<322 

— a 22 

— a 24 

0 

0 

X-a33 

— a 34 

0 

0 

0 

A —1344 

<ll)(A 

-<*22) (A 

-<*33) (A 

-CI 44 ) 


Thus, the characteristic equation is 

(A-flii)(A-fl^)(A-L3rO(A-fl44) = 0 

and the eigenvalues are 

A = fln, A = < 277 , X — a33, X = a^ 
which are precisely the diagonal entries of A. 


The following general theorem should be evident from the computations in the preceding example. 

THEOREM 7.1.1 


If A is an n triangular matrix (upper triangular, lower triangular, or diagonal), then the eigenvalues of A are the entries 

on the main diagonal of A. 










EXAMPLE 4 Eigenvalues of a Lower Triangular Matrix 


By inspection, the eigenvalues of the lower triangular matrix 


A = 




0 

2 

3 

-3 


are A = 



2 

3’ 


and A = 


I 

4' 


0 

0 

1 

4 


Complex Eigenvalues 


It is possible for the characteristic equation of a matrix with real entries to have complex solutions. In fact, because the 
eigenvalues of an n x « matrix are the roots of a polynomial of precise degree n, every nx.n matrix has exactly n eigenvalues if 
we count them as we count the roots of a polynomial (meaning that they may be repeated, and may occur in complex conjugate 
pairs). For example, the characteristic polynomial of the matrix 


A = 


-2 

5 


- 1 

2 


is 


det(A/ — A) — det 


A 4- 2 
— 5 


1 

A— 2 



I- 1 


so the characteristic equation is A 2 + 1 = 0> the solutions of which are the imaginary numbers A = i and A = — i- Thus we are 
forced to consider complex eigenvalues, even for real matrices. This, in turn, leads us to consider the possibility of complex 
vector spaces—that is, vector spaces in which scalars are allowed to have complex values. Such vector spaces will be 
considered in Chapter 10. For now, we will allow complex eigenvalues, but we will limit our discussion of eigenvectors to the 
case of real eigenvalues. 


The following theorem summarizes our discussion thus far. 


THEOREM 7.1.2 

Equivalent Statements 

If A is an nxn matrix and A is a real number, then the following are equivalent. 

(a) A is an eigenvalue of A. 

(b) The system of equations (A I — A) x = 0 has nontrivial solutions. 

(c) There is a nonzero vector x in R n such that j4x — Ax- 

(d) A is a solution of the characteristic equation det (A/ — A) = 0- 










Finding Eigenvectors and Bases for Eigenspaces 

Now that we know how to find eigenvalues, we turn to the problem of finding eigenvectors. The eigenvectors of A 
corresponding to an eigenvalue A are the nonzero vectors x that satisfy AN = Ax- Equivalently, the eigenvectors corresponding to 
A are the nonzero vectors in the solution space of (A/ — Al) x = 0—that is, in the null space of A I — A- We call this solution space 
the eigenspace of A corresponding to A- 


Eigenvectors and Bases for Eigenspaces 


Find bases for the eigenspaces of 


A = 


0 0 
1 2 
1 0 



Solution 

The characteristic equation of matrix A is A 3 — 5A 2 4 - 8 A — 4 = 0> or, in factored form, (A — 1) (A — 2 ) 2 = 0 (verify); thus the 
eigenvalues of A are Ai = 1 and A2 3 = 2, so there are two eigenspaces of A. 


By definition, 


*1 


x = 


*2 


*3 


is an eigenvector of A corresponding to A if and only if x is a nontrivial solution of (A I — A)x = 0—that is, of 


A 

- 1 

0 

A — 2 

2 

- 1 

'*f 

X 2 

_ 

'O' 

0 

- 1 

0 

A — 3 

X 3 


0 


If A = 2> then 3 becomes 


2 

0 

2' 

'*r 


'O' 

-1 

0 

-1 

X2 

= 

0 

-1 

0 

-1 

X3 


0 


Solving this system using Gaussian elimination yields (verify) 

x\ = -s, X2 = t, X2 = s 


Thus, the eigenvectors of A corresponding to A = 2 are the nonzero vectors of the form 



" — S~ 


" — s~ 


'o' 


' - r 


'o' 

X = 

t 

= 

0 

+ 

t 

= s 

0 

+t 

1 


s 


s 


0 


1 


0 


Since 


' - r 


'o' 

0 

and 

1 

1 


0 


are linearly independent, these vectors form a basis for the eigenspace corresponding to A = 2 - 


(3) 


If A = 1 , then 3 becomes 


































Solving this system yields (verify) 


1 

0 

2 

■*r 


0 

-1 

-1 

- 1 

*2 

= 

0 

-1 

0 

-2 

*3 


0 


*1 = - 2s, X 2 = s, *3 = s 
Thus the eigenvectors corresponding to \ = 1 are the nonzero vectors of the form 


— 2s 

S 

= s 

' - 2' 
1 

so that 

'-2 

1 

s 


1 


1 


is a basis for the eigenspace corresponding to A = 1 • 


♦ 


Notice that the zero vector is in every eigenspace, although it isn’t an eigenvector. 

Powers of a Matrix 

Once the eigenvalues and eigenvectors of a matrix A are found, it is a simple matter to find the eigenvalues and eigenvectors of 
any positive integer power of A; for example, if A is an eigenvalue of A and x is a corresponding eigenvector, then 

A 2 * = A(Ax) = A(Ax) = A(Ax) = A(Ax) = A 2 x 

which shows that a 2 is an eigenvalue of jf* and thatx is a corresponding eigenvector. In general, we have the following result. 


THEOREM 7.1.3 


If k is a positive integer, \ is an eigenvalue of a matrix A, andx is a corresponding eigenvector, then is an eigenvalue of 
and x is a corresponding eigenvector. 


Using Theorem 7.1.3 


In Example 5 we showed that the eigenvalues of 


A = 


0 0-2 
1 2 1 

1 0 3 


: 2 7 = 

128 and A = 

i 7 = 

' - r 


'o' 

0 

and 

1 

i 


0 


are eigenvectors of A corresponding to the eigenvalue A = 2> so from Theorem 7.1.3, they are also eigenvectors of 
corresponding to A = 2 7 = 128- Similarly, the eigenvector 

“ - 2 

1 
1 

of A corresponding to the eigenvalue A = 1 is also an eigenvector of corresponding to a = l 7 = 1- 


Eigenvalues and Invertibility 

























The next theorem establishes a relationship between the eigenvalues and the invertibility of a matrix. 


THEOREM 7.1.4 


A square matrix A is invertible if and only if\ = 0 is not an eigenvalue of A. 


Assume that A is an « x n matrix and observe first that A = 0 is a solution of the characteristic equation 


A” + cjA” 1 H-f c H = 0 

if and only if the constant term c„ is zero. Thus it suffices to prove that A is invertible if and only if 
c„*0. But 

det(A/ - A) = A” 4- c iA ” -1 + - 4- 


or, on setting a = 0 , 


det( — A)=c n or ( — 1) M det(j4) = c H 


It follows from the last equation that det(^) = 0 if and only if Cyi = 0 , and this in turn implies that A is 
invertible if and only if c „ * 0- 


Using Theorem 7.1.4 


The matrix A in Example 5 is invertible since it has eigenvalues A = 1 and A = 2, neither of which is zero. We leave it for the 
reader to check this conclusion by showing that det(j 4 ) ^ 0 - 

♦ 


Summary 

Theorem 7.1.4 enables us to add an additional result to Theorem 6.4.5. 


THEOREM 7.1.5 


Equivalent Statements 

If A is an nxn matrix, and if T 7?” —> Rf is multiplication by A, then the following are equivalent. 

(a) A is invertible. 


(b) ,4x = 0 has only the trivial solution. 


(c) The reduced row-echelon form of A is J n . 







(d) A is expressible as a product of elementary matrices. 

(e) j 4 x = b is consistent for every n x 1 matrix b- 

(f) = b has exactly one solution for every «xl matrix b- 

(g) det(j4) * 0- 

(h) The range of is R n . 

(i) Tj\ is one-to-one. 

(j) The column vectors of A are linearly independent. 

(k) The row vectors of A are linearly independent. 

(l) The column vectors of A span Bf 1 . 

(m) The row vectors of A span R n . 

(n) The column vectors of A form a basis for R n . 

(o) The row vectors of A form a basis for R n . 

(p) A has rank n. 

(q) A has nullity 0. 

(r) The orthogonal complement of the nullspace of A is R n . 

(s) The orthogonal complement of the row space of A is {Of 

(t) A 7 A is invertible. 

(u) A = 0 is not an eigenvalue of A. 



This theorem relates all of the major topics we have studied thus far. 


Exercise Set 7.1 



Click here for Just Ask! 


Find the characteristic equations of the following matrices: 

1 . 


(a) 


3 0 

8 - 1 


(b) 


10 -9 
4 -2 


(c) 


0 3 
4 0 


(d) 



-7 

2 


(e) 


0 0 
0 0 


(f) 


1 0 
0 1 


Find the eigenvalues of the matrices in Exercise 1. 

2 . 


Find bases for the eigenspaces of the matrices in Exercise 1. 

3. 


Find the characteristic equations of the following matrices: 

4 . 


(a) 


4 0 1 
-2 1 0 
-2 0 1 


(b) 


3 

1 

5 

1 


0 -5 
-1 0 

1 -2 




















(d) [-1 0 f 

-1 3 0 

-4 13 -1 

(e) [ 5 0 f 

1 1 0 
-7 1 0 


(f) 5 6 2 

0-1 -8 
1 0 -2 


Find the eigenvalues of the matrices in Exercise 4. 

5. 


Find bases for the eigenspaces of the matrices in Exercise 4. 

6 . 


Find the characteristic equations of the following matrices: 

7. 


(a) 0 0 

2 0 

1 0 

1 0 

0 1 

-2 0 

0 0 

0 1 

(b) f10 - 

9 i 


4-200 
0 0-2-7 

0 0 12 


Find the eigenvalues of the matrices in Exercise 7. 

8 . 


Find bases for the eigenspaces of the matrices in Exercise 7. 

9. 


By inspection, find the eigenvalues of the following matrices: 

10 . 







(b) 


3 0 0 
-2 7 0 

4 8 1 



0-^00 
0 0 10 

0 0 0 I 


Find the eigenvalues of for 
fl 3 7 111 


0 0 0 4 

0 0 0 2 


12 . 


Find the eigenvalues and bases for the eigenspaces of A 1 ^ for 



-2 -2 

2 1 

- 1 0 


13. 


Let A be a 2 x 2 matrix, and call a line through the origin of g} invariant under A if j4 x lies on the line when x does. Find 
equations for all lines in g}, if any, that are invariant under the given matrix. 


(a) 

(b) 

(c) 


A = 


A = 


A = 


4 -1 
2 1 


0 1 

-1 0 


2 3 
0 2 


Find det(A) given that A has ^(A) as its characteristic polynomial. 

14 . 


(a) ;?(A)=A 3 -2A 2 + A+5 


(b) p(X) =A 4 -A 3 1-7 



Hint See the proof of Theorem 7.1.4. 


Let A be an n x n matrix. 

15. 


(a) Prove that the characteristic polynomial of A has degree n. 


(b) Prove that the coefficient of A” in the characteristic polynomial is 1. 


16. 


Show that the characteristic equation of a 2 x 2 matrix A can be expressed as A — tr(A)A H det(A) = 0, where tr(A) is the 
trace of A. 


17. 


Use the result in Exercise 16 to show that if 


then the solutions of the characteristic equation of A are 
A = ^0 f d) _L )[(a — <a ?) 2 4- 4 ~bc 
Use this result to show that A has 


(a) two distinct real eigenvalues if (a — d) 2 I Abe > 0 


(b) two repeated real eigenvalues if (a — d) 2 A Abe = 0 


(c) complex conjugate eigenvalues if (a — d) 2 A- Abe < 0 


18. 


Let A be the matrix in Exercise 17. Show that if (a — d) 2 I Abe > 0 and b 0 , then eigenvectors of A corresponding to the 
eigenvalues 


Ai = A- {a d) A-Abe and X 2 = ^ (a I d) — /(a - tj ?) 2 + 4ic" 


are 


-b 
a — X\ 


and 


-b 

a-X.2 


respectively. 

Prove: If a, b, c, and d are integers such that a 4 b = c 4 d, then 

19 - r 

A= a b 
[c d_ 

has integer eigenvalues—namely, Aj = a 4 b and A 2 = a — e- 

Prove: If A is an eigenvalue of an invertible matrix A, and x is a corresponding eigenvector, then 1 / A is an eigenvalue of 
20 . g-1 s and x is a corresponding eigenvector. 



Prove: If A is an eigenvalue of A, x is a corresponding eigenvector, and s is a scalar, then A — s is an eigenvalue of A — si, 
21 . and x is a corresponding eigenvector. 


22 . 


Find the eigenvalues and bases for the eigenspaces of 


A = 


-2 2 3 
-2 3 2 
-4 2 5 


Then use Exercises 20 and 21 to find the eigenvalues and bases for the eigenspaces of 


(a) A" 1 

(b) A —31 

(c) A+21 


23. 

(a) Prove that if A is a square matrix, then A and a t have the same eigenvalues. 

Hint Look at the characteristic equation det(A/ — A) = 0- 

(b) Show that A and A T need not have the same eigenspaces. 

Hint Use the result in Exercise 18 to find a 2x2 matrix for which A and yj 7 " have different eigenspaces. 


Discussion 

Discovery 


Indicate whether each statement is always true or sometimes false. Justify your answer by giving 
24. a logical argument or a counterexample. In each part, A is an n x n matrix. 


(a) If Ax = Ax for some nonzero scalar A, then x is an eigenvector of A. 

(b) If A is not an eigenvalue of A, then the linear system (A/ — A) x = 0 has only the trivial 
solution. 


(c) A = 0 is an eigenvalue of A, then a 2 is singular. 

(d) If the characteristic polynomial of A is /’(A) = A H + 1, then A is invertible. 


25 . 


Suppose that the characteristic polynomial of some matrix A is found to be 
p (A) = (A — 1) (A — 3) (A — 4) . In each part, answer the question and explain your reasoning. 






(a) What is the size of A? 


(b) Is A invertible? 

(c) How many eigenspaces does A have? 


The eigenvectors that we have been studying are sometimes called right eigenvectors to 
26. distinguish them from left eigenvectors, which are n x 1 column matrices x that satisfy 

x 7 A = fix T for some scalar /.a. What is the relationship, if any, between the right eigenvectors and 
corresponding eigenvalues A of A and the left eigenvectors and corresponding eigenvalues of A? 
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7 a In this section we shall be concerned with the problem of finding a basis for 

R n that consists of eigenvectors of a given nxn matrix A. Such bases can be 
DIAGONALIZATION used to study geometric properties of A and to simplify various numerical 

computations involving A. These bases are also of physical significance in a 
wide variety of applications, some of which will be considered later in this text. 


The Matrix Diagonalization Problem 

Our first objective in this section is to show that the following two problems, which on the surface seem quite different, are 
actually equivalent. 


The Eigenvector Problem Given an n x « matrix A, does there exist a basis for R n consisting of eigenvectors of A? 


Given an HX « matrix A, does there exist an invertible matrix P such that p~^AP 

is a diagonal matrix? 


The latter problem suggests the following terminology. 


DEFINITION 


A square matrix A is called diagonalizable if there is an invertible matrix P such that P~^AP is a diagonal matrix; the matrix 
P is said to diagonalize A. 


The following theorem shows that the eigenvector problem and the diagonalization problem are equivalent. 

THEOREM 7.2.1 


If A is an n matrix, then the following are equivalent. 
(a) A is diagonalizable. 


(b) A has n linearly independent eigenvectors. 


Since A is assumed diagonalizable, there is an invertible matrix 









P 11 P 12 . P In 

7>21 i?22 - Pin 


P h1 ^h2 ■" Pnn 


such that p~ x ap is diagonal, say p~ l AP = D, where 

"Ai 0 - 0 

D= 0 a 2 ... 0 
0 0 a h 

It follows from the formula p~‘ [ AP = D that ap = ad ; that is, 



>11 

P 12 ■ 

- Pin ' 

'Ai 

0 ■ 

- 0 ' 


APll 

API! ' 

" ^nP 1h 

AP = 

P21 

^22 ‘ 

'• P2n 

0 

A 2 ■ 

•• 0 

= 

Al^21 

AP22 ‘ 

" ^-nP 2n 


Pnl 

Pn2 ■ 

" Pnn 

0 

0 ■ 

■■ A h 


APnl 

APn2 ' 

^nPnn 


If we now let pi, P 2 , ■■■, p H denote the column vectors of P, then from 1, the successive columns of ap 
are Aipi, A 2 p 2 , A H p„. However, from Formula 6 of Section 1.3, the successive columns of ap are App 
Ap 2 i—f An- Thus we nriust have 


An = Avu An = A 2 P 2 ,Ap H = a„p „ 


( 2 ) 


Since P is invertible, its column vectors are all nonzero; thus, it follows from 2 that Ap a 2 , ..., A„ are 
eigenvalues of A, and pi, P 2 , ■■■, p H are corresponding eigenvectors. Since P is invertible, it follows 
from Theorem 7.1.5 that pi, p 2 , ..., p„ are linearly independent. Thus A has n linearly independent 
eigenvectors. 

Assume that A has n linearly independent eigenvectors, pj , p 2 , , p„, with corresponding eigenvalues Ap Ap A„. 

and let 


P = 


P 11 P 12 - P In 
P21 P 22 ” P2n 


P h1 Pnl Pnn 

be the matrix whose column vectors are pp p 2 , ..., p H . By Formula 6 of Section 1.3, the column vectors of the product AP are 

'h 

But 


Apt, Ap2,.... Ap, 
Api = Ann, Ap 2 = A 2 p 2 , 


Ap « — A H p H 


so 


AP = 


Apii AP12 

AP21 A 2P22 


APln 

AP2n 


Ai^mI A 2 ^ h2 A nPnyi 



>11 P12 ' 

” P 1h" 

'Ai 

0 ■ 

- 0 ' 

= 

P21 P22 ‘ 

“ P2n 

0 

A 2 ■ 

•• 0 


Pnl Pn2 ‘ 

,J Pnn 

0 

0 ■ 

■■ A h 

where D is the diagonal matrix having the eigenvalues Ap A 2 > • • • 


on the main 


( 3 ) 


= PD 


linearly independent, P is invertible. Thus 3 can be rewritten as P~^AP = L’> that is, A is diagonalizable. 


Procedure for Diagonalizing a Matrix 




















The preceding theorem guarantees that an « x n matrix A with n linearly independent eigenvectors is diagonalizable, and the 
proof provides the following method for diagonalizing A. 


Step 1 . Find n linearly independent eigenvectors of A , say p i , p 2 , ..., p n . 

Step 2. Form the matrix P having pi , p 2 , p H as its column vectors. 

Step 3. The matrix P~^AP will then be diagonal with Aj, A^ • • • ,A H as its successive diagonal entries, where A; is the 
eigenvalue corresponding to p, for j = T 2, 


In order to carry out Step 1 of this procedure, one first needs a way of determining whether a given nxn matrix A has n linearly 
independent eigenvectors, and then one needs a method for finding them. One can address both problems at the same time by 
finding bases for the eigenspaces of A. Later in this section, we will show that those basis vectors, as a combined set, are linearly 
independent, so that if there is a total of n such vectors, then A is diagonalizable, and the n basis vectors can be used as the 
column vectors of the diagonalizing matrix P. If there are fewer than n basis vectors, then A is not diagonalizable. 


Finding a Matrix PThat Diagonalizes a Matrix A 


Find a matrix P that diagonalizes 


A = 


0 0-2 
1 2 1 

1 0 3 


Solution 


From Example 5 of the preceding section, we found the characteristic equation of A to be 

(A- l)(A-2) 2 = 0 


and we found the following bases for the eigenspaces: 



-1 


0 


-2 

A = 2: 1M — 

0 

1 

P2 = 

1 

0 

A = 1: 1'3 = 

1 

1 


There are three basis vectors in total, so the matrix A is diagonalizable and 

^10 -2 
P= ' 


diagonalizes A. As a check, the reader should verify that 


P~ X AP = 


"10 2 

-1 

O 

O 

1 

ro 

'-1 0 -2' 


'2 0 O' 

1 1 1 

1 2 1 

0 1 1 

= 

0 2 0 

- 1 0 - 1 

1 0 3 

1 0 1 


0 0 1 


♦ 


There is no preferred order for the columns of P. Since the ith diagonal entry of P~^AF is an eigenvalue for the ith column 
vector of P, changing the order of the columns of P just changes the order of the eigenvalues on the diagonal of P~^AP ■ Thus, 






















if we had written 


in Example 1, we would have obtained 


P = 


1 -2 0 

0 1 1 

1 1 0 


P~ l AP 


2 0 0 
0 1 0 
0 0 2 


A Matrix That Is Not Diagonalizable 


Find a matrix P that diagonalizes 


A = 


1 0 0 
1 2 0 
-3 5 2 


Solution 

The characteristic polynomial of A is 


so the characteristic equation is 


det(A/ — j4) 


A- 1 0 0 

-1 A— 2 0 

3 -5 A — 2 


= (A-1)(A— 2) 2 


(A — 1) (A — 2 ) 2 = 0 


Thus the eigenvalues of A are Aj = 1 and A 2 3 = 2. We leave it for the reader to show that bases for the eigenspaces are 




l" 

s 



'o' 

A= 1: 

PI = 

1 

s 

1 

II 

■< 

P 2 = 

0 

1 


Since A is a 3 x 3 matrix and there are only two basis vectors in total, A is not diagonalizable. 


Alternative Solution 


If one is interested only in determining whether a matrix is diagonalizable and is not concerned with actually finding a 
diagonalizing matrix P, then it is not necessary to compute bases for the eigenspaces; it suffices to find the dimensions of the 
eigenspaces. For this example, the eigenspace corresponding to \ = 1 is the solution space of the system 


0 

0 

o' 

■*f 


'O' 

- 1 

-1 

0 

*2 

= 

0 

3 

-5 

-1 

*3 


0 


The coefficient matrix has rank 2(verify). Thus the nullity of this matrix is 1 by Theorem 5.6.3, and hence the solution space is 
one-dimensional. 


The eigenspace corresponding to A = 2 is the solution space of the system 






















This coefficient matrix also has rank 2 and nullity 1 (verify), so the eigenspace corresponding to A = 2 is also one-dimensional. 
Since the eigenspaces produce a total of two basis vectors, the matrix A is not diagonalizable. 

♦ 


There is an assumption in Example 1 that the column vectors of P, which are made up of basis vectors from the various 
eigenspaces of A, are linearly independent. The following theorem addresses this issue. 


THEOREM 7.2.2 


Ifv 1 , \ r 2 > ..., v;t are eigenvectors of A corresponding to distinct eigenvalues Aj, A. 2 - • ••> Afc, then {v\, \' 2 , .... v;J is a linearly 
independent set. 


Let vj, v r 2 , ..., v;t be eigenvectors of A corresponding to distinct eigenvalues Aj, A 2 > • • •> Afc- We shall assume that vj, V 2 , 
..., vfc are linearly dependent and obtain a contradiction. We can then conclude that vj, V 2 , ..., are linearly independent. 

Since an eigenvector is nonzero by definition, ( Vl ) is linearly independent. Let r be the largest integer such that 
(vi, \’ 2 ,..., Vj.) is linearly independent. Since we are assuming that (y 1; V 2 ,..., v^} is linearly dependent, r satisfies 1 <r<k- 
Moreover, by definition of r, ( Vli V 2 , ) is linearly dependent. Thus there are scalars c\,ci, •••, not all zero, such 

that 


civi + C1V2 H-hCj.-i-iVj.-i-i =0 


Multiplying both sides of 4 by A and using 

Av 1 =Aivi, 


we obtain 


Av 7 = by?.,—. 


— A r _i_i 


(4) 


ciAivi fc 2 A 2 v 2 I-1- Cr+iAj.+iVj.+i = 0 

Multiplying both sides of 4 by A r +i and subtracting the resulting equation from 5 yields 

ci (Al - \y+] )vi -I- Cjf b. - Ar+I )V2 -I-1- Cy{\y - A ? + 1 )v,. = 0 

Since (y 1; y 2i y,,} is a linearly independent set, this equation implies that 

ci(Ai — A r+ i) = c 2 (A2 — A r+ i) =- = Cj.(Aj. — Aj. + i) = 0 
and since Ai, A. 2 » •••, A r +i are distinct by hypothesis, it follows that 

Cl —C2 — - — Cy — 0 


Substituting these values in 4 yields 

Since the eigenvector y,. +1 is nonzero, it follows that 


Cr+lVr+1 =0 

Cj-+1 = 0 


(5) 


( 6 ) 


( 7 ) 


ci, Cj-_|_ 1 


Equations 6 and 7 contradict the fact that 


are not all zero; this completes the proof. 









Theorem 7.2.2 is a special case of a more general result: Suppose that Ap A 2 , ■ • •, A^ are distinct eigenvalues and that 
we choose a linearly independent set in each of the corresponding eigenspaces. If we then merge all these vectors into a single 
set, the result will still be a linearly independent set. For example, if we choose three linearly independent vectors from one 
eigenspace and two linearly independent vectors from another eigenspace, then the five vectors together form a linearly 
independent set. We omit the proof. 


As a consequence of Theorem 7.2.2, we obtain the following important result. 


THEOREM 7.2.3 


If an n matrix A has n distinct eigenvalues, then A is diagonalizable. 


Proof Ifvi,v 2 , ..., v H are eigenvectors corresponding to the distinct eigenvalues Ap A 2 > • • -,A H > then by Theorem 7.2.2, vp v 2 , 
..., v H are linearly independent. Thus A is diagonalizable by Theorem 7.2.1. 


Using Theorem 7.2.3 


We saw in Example 2 of the preceding section that 



1 0 
0 1 
-17 8 


has three distinct eigenvalues: A = 4, A= 2 I \j~3' an< J A = 2 — 

4 


P~ [ AP = 


Therefore, A is diagonalizable. Further, 

0 0 
2+ft 0 

0 2-/3 


for some invertible matrix P. If desired, the matrix P can be found using the method shown in Example 1 of this section. 


♦ 


EXAMP A Diagonalizable Matrix 


From Theorem 7.1.1, the eigenvalues of a triangular matrix are the entries on its main diagonal. Thus, a triangular matrix with 
distinct entries on the main diagonal is diagonalizable. For example, 

'-12 4 O' 

0 3 1 7 

0 0 5 8 

0 0 0 -2 


♦ 


is a diagonalizable matrix. 













EXAMPLE 5 Repeated Eigenvalues and Diagonalizability 


It's important to note that Theorem 7.2.3 says only that if a matrix has all distinct eigenvalues (whether real or complex), then it 
is diagonalizable; in other words, only matrices with repeated eigenvalues might be nondiagonalizable. For example, the 3x3 
identity matrix 


h = 


l 

o 

o 


0 

1 

0 


0 

0 

1 


has repeated eigenvalues Aj 2 3 = 1 but is diagonalizable since any nonzero vector in p} is an eigenvector of the 3x3 identity 
matrix (verify), and so, in particular, we can find three linearly independent eigenvectors. The matrix 


J 3 = 


1 

0 

0 


1 

1 

0 


0 

1 

1 


also has repeated eigenvalues Ai 2 3 = 1> but solving for its eigenvectors leads to the system 

'0 1 O' 

(A/-J 3 )x = 0 0 1 x = 0 
0 0 0 

the solution of which is jq = t, *2 = 0> x 3 = 0- Thus every eigenvector of is a multiple of 

1' 

0 
0 

which means that the eigenspace has dimension 1 and that j 3 is nondiagonalizable. 


♦ 


Matrices that look like the identity matrix except that the diagonal immediately above the main diagonal also has l's on it, such 
as or are known as Jordan block matrices and are the canonical examples of nondiagonalizable matrices. The Jordan block 
matrix j has an eigenspace of dimension 1 that is the span of ej. These matrices appear as submatrices in the Jordan 
decomposition, a sort of near-diagonalization for nondiagonalizable matrices. 


Geometric and Algebraic Multiplicity 

We see from Example 5 that Theorem 7.2.3 does not completely settle the diagonalization problem, since it is possible for an 
nxn matrix A to be diagonalizable without having n distinct eigenvalues. We also saw this in Example 1, where the given 3x3 
matrix had only two distinct eigenvalues and yet was diagonalizable. What really matter for diagonalizability are the dimensions 
of the eigenspaces—those dimensions must add up to n in order for an nxn matrix to be diagonalizable. Examples Example 1 
and Example 2 illustrate this; the matrices in those examples have the same characteristic equation and the same eigenvalues, 
but the matrix in Example 1 is diagonalizable because the dimensions of the eigenspaces add to 3, and the matrix in Example 2 
is not diagonalizable because the dimensions only add to 2. The 3x3 matrices in Example 5 also have the same characteristic 
polynomial (A — l) 3 and hence the same eigenvalues, but the first matrix has a single eigenspace of dimension 3 and so is 
diagonalizable, whereas the second matrix has a single eigenspace of dimension 1 and so is not diagonalizable. 

A full excursion into the study of diagonalizability is left for more advanced courses, but we shall touch on one theorem that is 
important to a fuller understanding of diagonalizability. It can be proved that if A n is an eigenvalue of A, then the dimension of 
the eigenspace corresponding to Aq cannot exceed the number of times that A — Ao appears as a factor in the characteristic 
polynomial of A. For example, in Examples Example 1 and Example 2 the characteristic polynomial is 

(A- 1)(A-2) 2 














Thus the eigenspace corresponding to A = 1 is at most (hence exactly) one-dimensional, and the eigenspace corresponding to 
A = 2 is at most two-dimensional. In Example 1 the eigenspace corresponding to A = 2 actually had dimension 2, resulting in 
diagonalizability,but in Example 2 that eigenspace had only dimension 1, resulting in nondiagonalizability. 

There is some terminology that is related to these ideas. If Aq is an eigenvalue of an n x n matrix A, then the dimension of the 
eigenspace corresponding to Aq is called the geometric multiplicity of Aq, and the number of times that A — Aq appears as a factor 
in the characteristic polynomial of A is called the algebraic multiplicity of A. The following theorem, which we state without 
proof, summarizes the preceding discussion. 


THEOREM 7.2.4 


Geometric and Algebraic Multiplicity 

If A is a square matrix, then 

(a) For every eigenvalue of A, the geometric multiplicity is less than or equal to the algebraic multiplicity. 

(b) A is diagonalizable if and only if for every eigenvalue, the geometric multiplicity is equal to the algebraic 
multiplicity. 


Computing Powers of a Matrix 

There are numerous problems in applied mathematics that require the computation of high powers of a square matrix. We shall 
conclude this section by showing how diagonalization can be used to simplify such computations for diagonalizable matrices. 

If A is an n x « matrix and P is an invertible matrix, then 

{P~ l AP) 2 = P~ l APP~ { AP = P~ l AIAP = P~ l A 2 P 
More generally, for any positive integer k, 

{P~ l AP) k = P~ l A k P (8) 

It follows from this equation that if A is diagonalizable, and p~^ j\p = £, is a diagonal matrix, then 

P~ l A k P=(P~ 1 AP) k = D k (9) 


Solving this equation for ^ k yields 


A k = PD k P 1 (10) 

This last equation expresses the kth power of A in terms of the Mi power of the diagonal matrix D. But pf is easy to compute, 



0 ■ 

•• 0 ' 



l 

0 ■ 

•• 0 

0 

d2 • 

•• 0 

, then 

D k = 

0 

jk 

“2 ‘ 

•• 0 

0 

0 • 




0 

0 • 

jk 
“ “H 







Power of a Matrix 


Use 10 to find j 4 13 , where 


A = 


0 0-2 
1 2 1 

1 0 3 


Solution 


We showed in Example 1 that the matrix A is diagonalized by 

-1 0 - 2 ' 


and that 


Thus, from 10, 


A n = PD n P~ l = 


? = 


0 1 
1 0 





2 0 0 

D = P 

~ 1 AP = 

0 2 0 




0 0 1 

-1 0 

-2' 

2 13 0 0 

0 1 

1 

0 

2 13 0 

1 0 

1 

_0 

0 1 

-8190 

0 


-16382" 

8191 

8192 


8191 

8191 

0 


16383 


13 


1 


0 
1 1 


-1 0 -1 


( 11 ) 


Remark With the method in the preceding example, most of the work is in diagonalizing A. Once that work is done, it can be 
used to compute any power of A. Thus, to compute j 4 1000 we need only change the exponents from 13 to 1000 in 11. 


Exercise Set 7.2 



Click here for Just Ask! 


1. 


Let A be a 6 x 6 matrix with characteristic equation A 2 (A — 1) (A — 2) 3 = 0. What are the possible dimensions for eigenspaces 
ofA? 


A = 


4 0 1 
2 3 2 
1 0 4 


2 . 


Let 





















(a) Find the eigenvalues of A. 


(b) For each eigenvalue A, find the rank of the matrix A I -A- 


(c) Is A diagonalizable? Justify your conclusion. 


In Exercises 3-7 use the method of Exercise 2 to determine whether the matrix is diagonalizable. 


3. i 


4. 1 -1 


) 1 2 


6. _i 


-4 13 


In Exercises 8-11 find a matrix P that diagonalizes A, and determine P _1 AP 


A - “U 12 

8. -20 17 


, 4 - 1 0 

9. 6 -1 


1 0 0 
10. A= 0 1 1 
0 1 1 


2 0-2 

11. A= 0 3 0 

0 0 3 

In Exercises 12-17 find the geometric and algebraic multiplicity of each eigenvalue, and determine whether A is diagonalizable. 
If so, find a matrix P that diagonalizes A, and determine j\p . 

'19 — 9 _6' 

12. A= 25 - 11 -9 

17 -9-4 









13. ,4 = 


-1 4 -2 
-3 4 0 

-3 1 3 


14. ,4 = 


5 0 
1 5 
0 1 


0 

0 

5 


15. A = 


0 0 
0 0 
3 0 


0 

0 

1 


16. 


A = 


-2 0 0 0 

0-200 
0 0 3 0 

0 0 13 


17. 


A = 


-2 0 0 0 

0-2 5-5 
0 0 3 0 

0 0 0 3 


18. 


Use the method of Example 6 to compute where 


A = 


1 0 

-1 2 


19. 


Use the method of Example 6 to compute J4 11 , where 


A = 


-1 7-1 

0 1 0 
0 15 -2 


20 . 


In each part, compute the stated power of 


A = 


1 -2 

0 -1 

0 0 


(a) J 4 1000 

(b) j4 -1000 

(c) ^2301 

(d) A~ 2301 


21 . 


Find A ” if n is a positive integer and 








Let 




22 . 



b 

d 


Show that: 

(a) A is diagonalizable if {a —d) 2 A Abe > 0. 


(b) A is not diagonalizable if (a — d) 2 + Abe < 0. 


23. 


24. 


Hint See Exercise 17 of Section 7.1. 

In the case where the matrix A in Exercise 22 is diagonalizable, find a matrix P that diagonalizes A. 
Hint See Exercise 18 of Section 7.1. 

Prove that if A is a diagonalizable matrix, then the rank of A is the number of nonzero eigenvalues of A. 


Discussion 

Discovery 


Indicate whether each statement is always true or sometimes false. Justify your answer by giving 
25. a logical argument or a counterexample. 


(a) A square matrix with linearly independent column vectors is diagonalizable. 

(b) If A is diagonalizable, then there is a unique matrix P such that p~ 1 j\p is a diagonal 
matrix. 

(c) If vi, V 2 » and V 3 come from different eigenspaces of A, then it is impossible to express V 3 
as a linear combination of vj and vj. 

(d) If A is diagonalizable and invertible, then yl _1 is diagonalizable. 

(e) If A is diagonalizable, then T is diagonalizable. 


Suppose that the characteristic polynomial of some matrix A is found to be 
p (A) = (A — 1) (A — 3) 2 (A — 4) 3 . In each part, answer the question and explain your reasoning. 


26 . 








(a) What can you say about the dimensions of the eigenspaces of A? 


(b) What can you say about the dimensions of the eigenspaces if you know that A is 
diagonalizable? 

(c) If (vj, V 2 , V 3 } is a linearly independent set of eigenvectors of A all of which correspond 
to the same eigenvalue of A, what can you say about the eigenvalue? 


27. (For Readers Who Have Studied Calculus) If A 2 > •••■> • • • is an infinite sequence of 

n x n matrices, then the sequence is said to converge to the nxn matrix A if the entries in the ith 
row and /th column of the sequence converge to the entry in the ith row and /'th column of A for 

all i and j. In that case we call A the limit of the sequence and write lim^_, +cx A k = A. The 

algebraic properties of such limits mirror those of numerical limits. Thus, for example, if P is an 
invertible nxn matrix whose entries do not depend on k, then km A k = A if and only if 

k —* +00 

ton P~ 1 A k P = P~ 1 AP. 

k —* +00 

(a) Suppose that A is an n x n diagonalizable matrix. Under what conditions on the 

eigenvalues of A will the sequence A, / 1 2 , ..., ■ ■ ■ converge? Explain your reasoning. 

(b) What is the limit when your conditions are satisfied? 


28. (For Readers Who Have Studied Calculus) If A\ + A 2 H_ \-A k + — is an infinite series of 

nxn matrices, then the series is said to converge if its sequence of partial sums converges to 
some limit A in the sense defined in Exercise 27. In that case we call A the sum of the series and 
write A = A\ A A 2 4 - 4 A k \- -■ 

(a) From calculus, under what conditions on x does the geometric series 

1 I- x 4 x 2 H- hx*+~ 

converge? What is the sum? 

(b) Judging on the basis of Exercise 27, under what conditions on the eigenvalues of A would 
you expect the geometric matrix series / 4 A A- A 2 4 — I A k 4 — to converge? Explain 
your reasoning. 

(c) What is the sum of the series when it converges? 


Show that the Jordan block matrix j has A = 1 as its only eigenvalue and that the corresponding 
29. eigenspace is span {e 1 ) • 
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7.3 


ORTHOGONAL 

DIAGONALIZATION 


In this section we shall be concerned with the problem of finding an 
orthonormal basis for R n with the Euclidean inner product consisting of 
eigenvectors of a given nun matrix A. Our earlier work on symmetric 
matrices and orthogonal matrices will play an important role in the 
discussion that follows. 


Orthogonal Diagonalization Problem 

As in the preceding section, we begin by stating two problems. Our goal is to show that the problems are equivalent. 

The Orthonormal Eigenvector Problem Given an « x « matrix A, does there exist an orthonormal basis for R n with the 
Euclidean inner product that consists of eigenvectors of the matrix A? 


The Orthogonal Diagonalization Problem (Matrix Form) Given an n >; n matrix A, does there exist an orthogonal matrix P 
such that the matrix p~^AP = P 7 AP is diagonal? If there is such a matrix, then A is said to be orthogonally diagonalizable 
and P is said to orthogonally diagonalize A. 

For the latter problem, we have two questions to consider: 

Which matrices are orthogonally diagonalizable? 

How do we find an orthogonal matrix to carry out the diagonalization? 

With regard to the first question, we note that there is no hope of orthogonally diagonalizing a matrix A unless A is 
symmetric (that is, A —A 7 )- To see why this is so, suppose that 


p 7 ap=d 


( 1 ) 


where P is an orthogonal matrix and D is a diagonal matrix. Since P is orthogonal, PP 7 = P 7 P = /, so it follows that 1 can 
be written as 


a=pdp t 


( 2 ) 


Since D is a diagonal matrix, we have D — D 7 - Therefore, transposing both sides of 2 yields 


7 1 7* T 7 1 ^ T T T 

A t =(PDP t ) =(P T ) d t p t = pdp t = a 


so A must be symmetric. 


Conditions for Orthogonal Diagonalizability 

The following theorem shows that every symmetric matrix is, in fact, orthogonally diagonalizable. In this theorem, and for 
the remainder of this section, orthogonal will mean orthogonal with respect to the Euclidean inner product on R n . 



THEOREM 7.3.1 


If A is an fix n matrix, then the following are equivalent. 

(a) A is orthogonally diagonalizable. 


(b) A has an orthonormal set of n eigenvectors. 


(c) A is symmetric. 


Since A is orthogonally diagonalizable, there is an orthogonal matrix P such that p~^AP is diagonal. As 
shown in the proof of Theorem 7.2.1, the n column vectors of P are eigenvectors of A. Since P is orthogonal, these column 
vectors are orthonormal (see Theorem 6.6.1), so A has n orthonormal eigenvectors. 

Assume that A has an orthonormal set of n eigenvectors p 2 ,..., p H ) • As shown in the proof of Theorem 7.2.1, 
the matrix P with these eigenvectors as columns diagonalizes A. Since these eigenvectors are orthonormal, P is orthogonal 
and thus orthogonally diagonalizes A. 


In the proof that (a) = 4 - (&), we showed that an orthogonally diagonalizable « x « matrix A is orthogonally 
diagonalized by an n x n matrix P whose columns form an orthonormal set of eigenvectors of A. Let D be the diagonal 
matrix 


D = P~ l AP 


Thus 


A = PDP~ l or A = PDP r 

since P is orthogonal. Therefore, 

*7’ T T T T 

A T =(PDP T ) =pd t p t = pdp t = a 

which shows that A is symmetric. 


The proof of this part is beyond the scope of this text and will be omitted. 


Note in particular that every symmetric matrix is diagonalizable. 

Symmetric Matrices 

Our next goal is to devise a procedure for orthogonally diagonalizing a symmetric matrix, but before we can do so, we need 
a critical theorem about eigenvalues and eigenvectors of symmetric matrices. 


THEOREM 7.3.2 


If A is a symmetric matrix, then 




(a) The eigenvalues of A are all real numbers. 


(b) Eigenvectors from different eigenspaces are orthogonal. 


'if (a) The proof of part (a), which requires results about complex vector spaces, is discussed in Section 10.6. 


Proof (b) Let vi and V 2 be eigenvectors corresponding to distinct eigenvalues Aj and A 2 of the matrix A. We want to show 
that V j - \'2 = 0- The proof of this involves the trick of starting with the expression Jyj. V2 . It follows from Formula 8 of 
Section 4.1 and the symmetry of A that 


Av 1 - V2 = vi - A 1 V2 = vi - Ax 2 


( 3 ) 


But vi is an eigenvector of A corresponding to Ai, and v 2 is an eigenvector of A corresponding to A 2 , 
so 3 yields the relationship 

Aivi - V2 = vi - A 2 V 2 


which can be rewritten as 


(At - A 2 ) (vi - v 2 ) = 0 


( 4 ) 


But Ai - A 2 * 0, since Ai and A 2 were assumed distinct. Thus it follows from 4 that Vl . V2 = 0- 


Remark We remind the reader that we have assumed to this point that all of our matrices have real entries. Indeed, we shall 
see in Chapter 10 that part (a) of Theorem 7.3.2 is false for matrices with complex entries. 

Diagonalization of Symmetric Matrices 

As a consequence of the preceding theorem we obtain the following procedure for orthogonally diagonalizing a symmetric 
matrix. 

Step 1. Find a basis for each eigenspace of A. 

Step 2. Apply the Gram-Schmidt process to each of these bases to obtain an orthonormal basis for each eigenspace. 

Step 3. Form the matrix P whose columns are the basis vectors constructed in Step 2; this matrix orthogonally 
diagonalizes A. 

The justification of this procedure should be clear: Theorem 7.3.2 ensures that eigenvectors from different eigenspaces are 
orthogonal, whereas the application of the Gram-Schmidt process ensures that the eigenvectors obtained within the same 
eigenspace are orthonormal. Therefore, the entire set of eigenvectors obtained by this procedure is orthonormal. 



An Orthogonal Matrix PThat Diagonalizes a Matrix A 


Find an orthogonal matrix P that diagonalizes 


A = 


4 2 2 
2 4 2 
2 2 4 


Solution 

The characteristic equation of A is 

det(A/ — A) = det 


A —4 -2 -2 
-2 A—4 -2 
— 2 -2 A —4 


= (A — 2) Z (A — 8 ) = 0 


Thus the eigenvalues of A are Ai 2 = 2 and = 3 . By the method used in Example 5 of Section 7.1, it can be shown that 


Ul - 

- f 
1 

and 

“2 = 

- f 
0 


0 



1 


(5) 


form a basis for the eigenspace corresponding to A = 2- Applying the Gram-Schmidt process to (m, u 2 } yields the 
following orthonormal eigenvectors (verify): 


vi - 

1 

1 

""^T| 

_1 

and \ r 2 = 

- 1 / 

-1 / {s 


0 


2! 


( 6 ) 


The eigenspace corresponding to A = 8 has 


113 = 


as a basis. Applying the Gram-Schmidt process to { U3 } yields 


v 3 : 


1 / 

1 / ft 
1 / ft 


Finally, using vq, V 2 , and V 3 as column vectors, we obtain 

-1//2 -ufe 1//3 


P = 


1//2 - 1//6 1//3 
0 2! y[e \iy[3 

which orthogonally diagonalizes A. (As a check, the reader may wish to verify that p T AP is a diagonal matrix.) 























Exercise Set 7.3 



Click here for Just Ask! 


Find the characteristic equation of the given symmetric matrix, and then by inspection determine the dimensions of the 
1* eigenspaces. 


(a) 


1 2 
2 4 


(b) 



-4 2 

1 -2 
-2 -2 


(c) 


1 1 
1 1 
1 1 


1 

1 

1 


(d) 


4 2 2 
2 4 2 
2 2 4 


(e) 


4 4 0 0 
4 4 0 0 
0 0 0 0 
0 0 0 0 


(f) 



- 1 0 0 

2 0 0 

0 2-1 
0-1 2 


In Exercises 2-9 find a matrix P that orthogonally diagonalizes A, and determine p -1 j±p. 


2 . 


A = 


3 1 
1 3 



2/3 



-2 

3 


4 . 





















5. A = 


-2 0 
0 -3 
-36 0 


-36 

0 

-23 


6. A — 


1 1 0 
1 1 0 
0 0 0 


7. A = 


2 

- 1 
- 1 


1 

2 

1 


A = 


3 10 0 
13 0 0 
0 0 0 0 
0 0 0 0 


A = 


-7 24 
24 7 

0 0 
0 0 


0 0 
0 0 
-7 24 
24 7 


10 . 


Assuming that b ^ 0> find a matrix that orthogonally diagonalizes 

ct b 
b a 


11 . 


Prove that if A is any mxn matrix, then j[ t a has an orthonormal set of n eigenvectors. 


12 . 


(a) Show that if v is any n x 1 matrix and / is the n x n identity matrix, then / _ yy T is orthogonally diagonalizable. 


(b) Find a matrix P that orthogonally diagonalizes / _ vv T if 


v = 


13. 


Use the result in Exercise 17 of Section 7.1 to prove Theorem 1.3.2a for 2 x 2 symmetric matrices. 


Discussion 

Discovery 


14 . 


Indicate whether each statement is always true or sometimes false. Justify your answer by 
giving a logical argument or a counterexample. 


















(a) If A is a square matrix, then and are orthogonally diagonalizable. 


(b) If vi and \'2 are eigenvectors from distinct eigenspaces of a symmetric matrix, then 
11 v 1 H v 2 11 2 = llvill 2 + llvill 2 - 

(c) An orthogonal matrix is orthogonally diagonalizable. 

(d) If A is an invertible orthogonally diagonalizable matrix, then ^ -1 is orthogonally 
diagonalizable. 


Does there exist a 3 x 3 symmetric matrix with eigenvalues Aj = — 1, A 2 = 3’ A 3 = 7 and 
15* corresponding eigenvectors 


0 " 


V 


' 0 " 

1 

? 

0 

7 

1 

-1 


0 


1 


If so, find such a matrix; if not, explain why not. 


Is the converse of Theorem 7.3.2 b true? 

16 . 
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Chapter 7 


Supplementary Exercises 


1. 


(a) Show that if 0 < 9 < then 


A = 


cos# —sin# 
sin# cos# 


has no eigenvalues and consequently no eigenvectors. 

(b) Give a geometric explanation of the result in part (a). 


2 . 


Find the eigenvalues of 


A = 


0 1 0 

0 0 1 

k 3 - 3k 2 3k 


(a) Show that if D is a diagonal matrix with nonnegative entries on the main diagonal, then there is a matrix S such 
that s 2 - D- 


(b) Show that if A is a diagonalizable matrix with nonnegative eigenvalues, then there is a matrix S such that S 2 — A- 


(c) Find a matrix S such that ?2 _ ^ if 


A = 


1 3 1 
0 4 5 
0 0 9 


Prove: If A is a square matrix, then A and T have the same characteristic polynomial. 


Prove: If A is a square matrix and p (,\) = det(A/ — A) is the characteristic polynomial of A, then the coefficient of \ K_1 
in p (A) is the negative of the trace of A. 


Prove: If b * 0, then 
a b 


6 . 











is not diagonalizable. 


In advanced linear algebra, one proves the Cayley-Hamilton Theorem , which states that a square matrix A satisfies its 
7. characteristic equation; that is, if 

cod C 1 A 4 c 2 A^ d~ ■■■ 4 c H _iA M + A H = 0 

is the characteristic equation of A, then 

cqI 4 c\A I c 2 j4^ + ■■■ 4 ^ H A^ = 0 

Verify this result for 



Exercises 8-10 use the Cayley-Hamilton Theorem, stated in Exercise 7. 

8 . 

(a) Use Exercise 16 of Section 7.1 to prove the Cayley-Hamilton Theorem for arbitrary 2x2 matrices. 


(b) Prove the Cayley-Hamilton Theorem for n x n diagonalizable matrices. 


The Cayley-Hamilton Theorem provides a method for calculating powers of a matrix. For example, if A is a 2 x 2 matrix 
9. with characteristic equation 

co 4 c 1 A 4 A = 0 

then CQ i hC{ A | A 2 = 0> so 


Multiplying through by A yields A 2 = —c\A 2 — cqA> which expresses A 2 in terms of a 2 and A, and multiplying through 
by a 2 yields a 4 = —c\A 2 — cqA 2 > which expresses a 4 in terms of A 2 and A 2 - Continuing in this way, we can calculate 
successive powers of A simply by expressing them in terms of lower powers. Use this procedure to calculate A 2 A 2 , A 4 , 
and A 2 for 



10 . 


Use the method of the preceding exercise to calculate a 2 and A 4 for 

'0 1 0 
A= 0 0 1 

1 -3 3 


Find the eigenvalues of the matrix 


11. 1 

~ c \ c 2 ■ 

,J c n 

11 

<=2 ■ 

" c n 


ci c 2 ■ 

- 1 

E 


12 . 


(a) It was shown in Exercise 15 of Section 7.1 that if A is an n x n matrix, then the coefficient of A” in the 
characteristic polynomial of A is 1. (A polynomial with this property is called monic .) Show that the matrix 

0 0 0 -0 
10 0-0 
0 1 0 - 0 

0 0 0 -1 

has characteristic polynomial p (\) =c 0 l ciA + - + c„_iA M_1 fA H -This shows that every 
monic polynomial is the characteristic polynomial of some matrix. The matrix in this 
example is called the companion matrix of p(\). 

Hint Evaluate all determinants in the problem by adding a multiple of the second row to the first to introduce a 
zero at the top of the first column, and then expanding by cofactors along the first column. 

(b) Find a matrix with characteristic polynomial p(X) — 1 — 2A h A 2 I 3 A 2 f- A^. 


A square matrix A is called nilpotent if A” = 0 for some positive integer n. What can you say about the eigenvalues of a 

13. nilpotent matrix? 

Prove: If A is an n x n matrix and n is odd, then A has at least one real eigenvalue. 

14. 

0, 1, and -1 with corresponding eigenvectors 
ll [O' 

-1 , 1 

lj [l 

respectively. 

16. SUPPOSe ^ a 4 X 4 matriX A haS d8enValUeS >1 = !• = - 2. A 3 = 3. and A4 = _ 3. 

(a) Use the method of Exercise 14 of Section 7.1 to find det(A)- 

(b) Use Exercise 5 above to find tr(A)- 

Let A be a square matrix such that A 3 — A- What can you say about the eigenvalues of A? 

17. 




Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 





Chapter 7 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Section 7.1 

Tl. (Characteristic Polynomial) Some technology utilities have a specific command for finding characteristic polynomials, and 
in others you must use the determinant function to compute det(A/ — A) - Read your documentation to deter mi ne which 
method you must use, and then use your utility to find ^(A) =0 for the matrix in Example 2. 


T2. (Solving the Characteristic Equation) Depending on the particular characteristic polynomial, your technology utility may 
or may not be successful in solving the characteristic equation for the eigenvalues. See if your utility can find the eigenvalues 
in Example 2 by solving the characteristic equation p, (A) = 0- 


T3. 

(a) Read the statement of the Cayley-Hamilton Theorem in Supplementary Exercise 7 of this chapter, and then use your 
technology utility to do that exercise. 

(b) If you are working with a CAS, use it to prove the Cayley-Hamilton Theorem for 3 x 3 matrices. 


T4. (Eigenvalues) Some technology utilities have specific commands for finding the eigenvalues of a matrix directly (though the 
procedure may not be successful in all cases). If your utility has this capability, read the documentation and then compute the 
eigenvalues in Example 2 directly. 


T5. (Eigenvectors) One way to use a technology utility to find eigenvectors corresponding to an eigenvalue \ is to solve the 
linear system (A/ — j4)x = 0- Another way is to use a command for finding a basis for the nullspace of XI — A (if available). 
However, some utilities have specific commands for finding eigenvectors. Read your documentation, and then explore 
various procedures for finding the eigenvectors in Examples 5 and 6. 


Section 7.2 

Tl. (Diagonalization) Some technology utilities have specific commands for diagonalizing a matrix. If your utility has this 
capability, read the documentation and then use your utility to perform the computations in Example 2. 

Note Your software may or may not produce the eigenvalues of A and the columns of P in the same order as the example. 



Section 7.3 


Tl. (Orthogonal Diagonalization) Use your technology utility to check the computations in Example 1. 
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CHAPTER 


Linear Transformations 


NTRODUCTION In Sections 4.2 and 4.3 we studied linear transformations from R n to R m . In this chapter we shall 
define and study linear transformations from an arbitrary vector space l/to another arbitrary vector space W. The results we 
obtain here have important applications in physics, engineering, and various branches of mathematics. 
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8.1 

GENERAL LINEAR 
TRANSFORMATIONS 


In Section 4.2 we defined linear transformations from R n to R m . In this 
section we shall extend this idea by defining the more general concept of a 
linear transformation from one vector space to another. 


Definitions and Terminology 

Recall that a linear transformation from Rf to R m was first defined as a function 

70i, X2, X„) = (wi, W2, W m ) 

for which the equations relating wq, vi?2»- and Jtq, X 2 ,--x n are linear. Subsequently, we showed that a transformation 
T\R n —» R m is linear if and only if the two relationships 

T(u + v) = T (u) + T (v) and T(cii) = cT(u) 

hold for all vectors u and v in R n and every scalar c (see Theorem 4.3.2). We shall use these properties as the starting point 
for general linear transformations. 


DEFINITION 


If 7 : y —* ffi is a function from a vector space V into a vector space W, then T is called a linear transformation from V 
to W if, for all vectors u and v 'mV and all scalars c, 

(a) T(„ + v ) = T(„) + T(v) 

(b) 7(cu)=cT(u) 

In the special case where Y — W, the linear transformation T\ V —> V is called a linear operator on V. 


Matrix Transformations 


Because the preceding definition of a linear transformation was based on Theorem 4.3.2, linear transformations from R n to 
R m , as defined in Section 4.2, are linear transformations under this more general definition as well. We shall call linear 
transformations from Rf to R™ matrix transformations, since they can be carried out by matrix multiplication. 


EXAMPLE 2 The Zero Transformation 















Let V and W be any two vector spaces. The mapping T: V -> W such that 7Y V ) = 0 for every v in V is a linear 

transformation called the zero transformation. To see that T is linear, observe that 

7(u + v)=0, T(u) = 0, T(v) = 0, and Tikn) = 0 


Therefore, 


T(u + v) = T(u) 4 T(y) and T(ku) = kT{ u) 


♦ 


The Identity Operator 


Let Tbe any vector space. The mapping iy —> V defined by /( v ) = v is called the identity operator on V. The verification 
that I is linear is left for the reader. 

♦ 


Dilation and Contraction Operators 


Let Vbe any vector space and k any fixed scalar. We leave it as an exercise to check that the function T\ V —» V defined by 

7(v) =jfcv 

is a linear operator on V. This linear operator is called a dilation of V with factor k if 4 > 1 and is called a contraction of V 
with factor k if 0 < k < 1 • Geometrically, the dilation “stretches” each vector in V by a factor of k, and the contraction of V 
“compresses” each vector by a factor of k (Figure 8.1.1). 



((j) Dilaluiu id' V 



Figure 8.1.1 














Orthogonal Projections 


In Section 6.4 we defined the orthogonal projection of R m onto a subspace W. [See Formula 6 and the definition preceding it 
in that section.] Orthogonal projections can also be defined in general inner product spaces as follows: Suppose that W is a 
finite-dimensional subspace of an inner product space V; then the orthogonal projection of V onto W is the transformation 
defined by 

TYv) = projf^v 

(Figure 8.1.2). It follows from Theorem 6.3.5 that if 

S= fwi, W 2 , 

is any orthonormal basis for W, then T(y) is given by the formula 

T(v) = pioj^v = (v, wi )wi 4- iy, w 2 )w 2 + - + (v, w r \w> 

The proof that T is a linear transformation follows from properties of the inner product. For example, 

7(u 4- v) — (u -f v, wi Jwi -I- (u + v, w 2 Jw 2 -1- ™ -1- (u -f v, 

= (u, wi Jwi -I- (u, w 2 Jw 2 H-f (u, 

-I- iv, wi Jwi -I- (v, w 2 Jw 2 H-h iv, Wj-JWj. 

= 7 (u) + T(v) 

Similarly, T(kii) =/c7(u). 



Figure 8.1.2 

The orthogonal projection of V onto W. 


♦ 


Computing an Orthogonal Projection 


As a special case of the preceding example, let y = p} have the Euclidean inner product. The vectors W i = (1, 0, 0) and 
t,v 2 = (0, 1, 0) form an orthonormal basis for the xy-plane. Thus, if v = (x, y, z) is any vector in pf, the orthogonal 
projection of fy onto the x>'-plane is given by 

7(y) = (v, wi Jwi 4- (v, w 2 Jw 2 
= x(l, 0, 0) 4^(0, 1, 0) 

= (*,y. 0 ) 


(See Figure 8.1.3.) 






H r. y. -) 



Figure 8.1.3 

The orthogonal projection of g? onto the xy-plane. 


♦ 


A Linear Transformation from a Space Vto R n 


Let S— {wi, w 2 ,..., w„} be a basis for an //-dimensional vector space V, and let 

!»,s'= (i i,k 2 ,..., k n ) 

be the coordinate vector relative to S of a vector v in V ; thus 

v = £iwi 4 k 2 vr 2 I-1- i H w « 

Define T: V —> to be the function that maps v into its coordinate vector relative to S —that is, 

T(v) = (y) s =(k h k 2 . k») 

The function T is a linear transformation. To see that this is so, suppose that u and v are vectors in V and that 

u = cjwi -1- c 2 w 2 \ -h c H w H and v = d pvi 4 d 2 w 2 H-h d? H w H 


Thus 

But 

so 

Therefore, 

(u + v) s = (u) 5- 4 (v) s and (in) s - k( n) s 

Expressing these equations in terms of T, we obtain 

T(u + v) = T(u) 4 T(y) and T(kn) = kT{ u) 
which shows that T is a linear transformation. 


(™)^= i c \’ c 2, and (v) s =(d h d 2 ,...,d„) 

u T v = (ei 4 ii)wi 4 (c 2 4 d 2 )ve 2 4 - 4 (c„ H d„) w„ 
kn= (kc\)w\ + (kc 2 )W 2 I-H 

(«+v) 5 =(q | d l ,c 2 + d 2 ,...,c„ + d„) 

(iu) s = ike 1 , kc 2 ,..., kc„) 


Remark The computations in the preceding example could just as well have been performed using coordinate vectors in 
column form; that is, 


[u + v] 5 -= [ 11 ] 5-4 [v]^ and [in] (? =i[u] 5 









EXAMPLE 8 A Linear Transformation from p n to P n+1 


Let p = p( x ) = cn + C1 x + ... + CyjX n be a polynomial in p n , and define the function T\P n —* P H+ i by 

T(p) = T(p(x)) = xp(x) =c$x + c\x 2 4- ■“ + c„x H+1 

The function Tis a linear transformation, since for any scalar k and any polynomials pi and \> j in p n we have 

^(lM + P2) = T (Pl(x) + P2(*)) =x(P\(x) +P2(*)) 

= xpi(x) + xp 2 (x) = T(m) 4- T{m) 


and 

T{kv) = Tikpix)) = x(kp(x)) = k(xp(x)) = kT( p) 
(Compare this to Exercise 4 of Section 4.4.) 


♦ 


EXAMPLE 9 A Linear Operator on p n 

Let p = p(x) — co 4- ci* 4-h c n x n be a polynomial in p n , and let a and b be any scalars. We leave it as an exercise to 

show that the function T defined by 

T(p) = T(p(x)) — p(ax + &) = co 4- c\(ax + &) 4-h c n (ax b) n 

is a linear operator. For example, if ax 4- b = 3x — 5, then p-p 2 _* Pj would be the linear operator given by the formula 

T(cn 4 C]X 4 c?x 2 ) =cn + ci (3x - 5) 4- c?(3x - 5) 2 

♦ 


EXAMPLE 10 A Linear Transformation Using an Inner Product 


Let V be an inner product space, and let yg be any fixed vector in V. Let T. V —* R be the transformation that maps a vector 
v into its inner product with yg—that is, 

r(v) = (v, v 0 } 

From the properties of an inner product. 

Till 4- v) = (u 4- v, vo} = (u, vo} + (v, vo} = T (u) + T (v) 
and 

T(kn) = (jfcii, vo} = k(n, vq} = kT (u) 


so T is a linear transformation. 


♦ 


A Linear Transformation from C 1 ( - oo , oo ) to F( — oo , oo) 


Calculus Required 













Let V — C 1 ( — oo , oo ) be the vector space of functions with continuous first derivatives on ( _ oo, oo ), and let 
W = F ( — oo , oo ) be the vector space of all real-valued functions defined on ( _ oo , oo )• Let V —► W be the 
transformation that maps a function f = j (*) into its derivative—that is, 

0(0 =/'(*) 

From the properties of differentiation, we have 

D{i + g) =D(i) + £)(g) and D(kf) = kD(f) 

Thus, D is a linear transformation. 


♦ 


EXAMPLE 12 A Linear Transformation from C( - oo , oo ) to C 1 (— oo , oo) 


Calculus Required 


Let V = C(— oo, oo)be the vector space of continuous functions on ( _ oo , oo ) ? and let W — C 1 ( — oo , oo ) be the 
vector space of functions with continuous first derivatives on ( _ oo , oo )• Let V — > W be the transformation that maps 

f = f (x) into the integral / f F° r example, if f = then 

h 


m = 



x 

n 


x 


3 


3 


From the properties of integration, 

^(f+g) 

J(cf) 


we have 

r \f(t) + g(t))dt= ff{t)dt + 
./o 


J 0 

=r 

./0 


cf {t)dt 


=<[’ 


f(t)dt = cJ(i) 



g{t)dt = J{i)l J{ g) 


so 7 is a linear transformation. 




EXAMPLE 13 A Transformation That Is Not Linear 


Let 7 : M nn _> R be the transformation that maps an n x n matrix into its determinant: 

T{A) = det(j4) 

If n > 1, then this transformation does not satisfy either of the properties required of a linear transformation. For example, we 
saw in Example 1 of Section 2.3 that 

det(74i -I- A 2 ) * det(jdi) -I- det(742) 
in general. Moreover, det(cTl) =c H det(j4), so 

det(cj4) * c det(j4) 


in general. Thus T is not a linear transformation. 


♦ 


Properties of Linear Transformations 


If T:V 


W is a linear transformation, then for any vectors vq and V 2 in V and any scalars c\ and we have 











T(civi + C2V 2 ) - rc^ivi) 4- T(c2'V2)=ciT(y 1 )+c 2 T(y 2 ) 
and, more generally, if v\ , V2,..., v H are vectors in V and cj, c 2, ..., c n are scalars, then 


r(civi +C2V2 + -“ + c M v M ) =cir(vi) +C27CV2) +- + c M r(v M ) 

Formula 1 is sometimes described by saying that linear transformations preserve linear combinations. 
The following theorem lists three basic properties that are common to all linear transformations. 

THEOREM 8.1.1 

IfT.V -* W is a linear transformation, then 

(a) 7(0) =0 

(b) T{ - v) = - T (v) for all v in V 

(c) 7(v - w) - 7(v) - T(yt)far all v and w in V 


Let v be any vector in V. Since Qv = we have 


7(0) = T(0v) = 07(v) =0 

which proves (a). Also, 

T( - v) = 7(( - l)v) = ( - l)7(v) = - 7(v) 
which proves ( b ). Finally, v -w = v+ (- l)w; thus 

7(v — w) = 7(v + ( — l)w) 

= 7(v) + ( - 1)7(w) 

= 7(v)-7(w) 

which proves (c). 

In words, part ( a ) of the preceding theorem states that a linear transformation maps 0 to 0. This property is useful for 
identifying transformations that are not linear. For example, if xg is a fixed nonzero vector in gf, then the transformation 

T(x) = x + x n 

has the geometric effect of translating each point x in a direction parallel to x 0 through a distance of || X q || (Figure 8.1.4). 
This cannot be a linear transformation, since 7(0) = xq> so T does not map 0 to 0. 




Figure 8.1.4 



T(x) = x 4- XQ translates each point x along a line parallel to xg through a distance || X q||. 


Finding Linear Transformations from Images of Basis Vectors 


Theorem 4.3.3 shows that if Tis a matrix transformation, then the standard matrix for T can be obtained from the images of 
the standard basis vectors. Stated another way, a matrix transformation is completely determined by its images of the 
standard basis vectors. This is a special case of a more general result: If T: V —* W is a linear transformation, and if 
(vi, v 2> ..., v } is any basis for V, then the image 7(y) of any vector v in V can be calculated from the images 

TM,T(w 7 T(y») 

of the basis vectors. This can be done by first expressing v as a linear combination of the basis vectors, say 

v = ciV! lc 2 v 2 H- 


and then using Formula 1 to write 


T(v) =ciT(vi) f c 2 T(\ 2 ) 4- ~‘ + c„T(y„) 


In words, a linear transformation is completely determined by the images of any set of basis vectors. 


Computing with Images of Basis Vectors 


Consider the basis S= (vi, v 2 , v 3 ) for R 2 , where Vl = (1, 1, 1), v 2 = (1, 1, 0), and V3 = (l ; 0, 0)- Let r-r 2 —* R 2 be the 
linear transformation such that 

TM = CL 0), T(v 2 ) = (2, - 1), T(y f) = (4, 3) 

Find a formula for T(x\, x 2 , x 3 ); then use this formula to compute 7(2, — 3, 5)- 


Solution 

We first express x = (xj, x 2 , x 3 ) as a linear combination of V j = (1, 1, 1), v 2 = (1, 1, 0), and v 3 = (1, 0, 0)- If we write 

(*1, x 2 . *3) = ci(l. 1. 1) T c 2 (l l 0) -I c 3 (l, 0, 0) 
then on equating corresponding components, we obtain 

Cl+C 2 + C 3 =X1 
C 1 + C 2 =X 2 

Cl =x 3 

which yields c\ = x 2 , c 2 = x 2 —x 2 , c 2 = x i — x 2 , so 

(xi, x 2 , x 2 ) =x 2 (l, 1, 1) + (x 2 -x 2 )(l, 1, 0) + (xi -x 2 )(l, 0, 0) 

= x 3 vi + (x 2 -x 2 )v 2 + (xi-x 2 )v 2 


Thus 








T(xi,X2,X3)=X3T(yi) 4 (x2~X3)T(y 2 ) 4 (*l - X2)T(y 3 ) 

= *3(1, 0) 4- (* 2 “*3)(2, - 1) + (*1 -x 2 ) (4, 3) 

— (4*1 - 2x2 -*3> 3*i -4x 2 413 ) 

From this formula, we obtain 

7(2, -3,5) = (9, 23) 

+ 

In Section 4.2 we defined the composition of matrix transformations. The following definition extends that concept to 
general linear transformations. 


DEFINITION 


If T\ : U —* V and T 2 \V —* W are linear transformations, then the composition of j 2 W ^ 1 7p denoted by T 2 o T\ 
(which is read “7 2 circle is the function defined by the formula 

(7 2 o7 1 )(u) = 7 2 (7 1 (u)) 

where a is a vector in U. 


Remark Observe that this definition requires that the domain of t 2 (which is V) contain the range of T \; this is essential for 
the formula T 2 (T\ ( 11 )) to make sense (Figure 8.1.5). The reader should compare 2 to Formula 18 in Section 4.2. 


T : ■- F, 



n T|(u) Ft(F,<u)) 

V V tv 

Figure 8.1.5 

The composition of t 2 with Ty 


The next result shows that the composition of two linear transformations is itself a linear transformation. 

THEOREM 8.1.2 

If Ty.U _ * V an d T 2 '.V _ ► W are linear transformations, then (7 2 o T\).U -> W is also a linear transformation. 


>of If u and v are vectors in U and c is a scalar, then it follows from 2 and the linearity of and T 2 that 

(7 2 o 70 (u 4 v) = 7 2 (7i (u 4 v)) = 7 2 (7 t (u) 4 T x (v)) 

= 7 2 (7 1 (u))4 7 2 (7 1 (v)) 

= (7 2 o70(u)4(7 2 o70(v) 














and 


(T 2 oTO (cn) = T 2 {T X (cn)) = 7 2 (^ (u)) 

= C 7 2 (r 1 (u))= e (T 2 oT 1 )(u) 

Thus 7 2 o T\ satisfies the two requirements of a linear transformation. 


EXAM Composition of Linear Transformations 


Let T\\P\ _* P 2 an d 7 2 :7 2 _* P 2 t * ie linear transformations given by the formulas 

T\(p(x))=xp(x) and T 2 (p(x)) = p(2x 4- 4) 

Then the composition (T 2 aT\)\Pi -* P 2 is given by the formula 

(T 2 o Ti)(p(x)) = TMTi (*(*))) = T 2 (xp(x)) = (2x + 4)p(2x + 4) 

In particular, if p ( x ) = c 0 4 - C \x, then 

(7 2 o Ti)(p(x)) = (T 2 aTi)(c 0 +cix) = (2x 4- 4)(cq T c\{2x + 4)) 

= co(2x + 4) 4- c\(2x 4- 4 ) 2 

♦ 


Composition with the Identity Operator 


If T: V -» V is any linear operator, and if 7 y -» y is the identity operator (Example 3), then for all vectors v in V , we 

have 

(7o/)(v) = 7(/(v)) = 7(v) 

(loT)(v)=I(T(v)) = T(v) 

It follows that To l and / 0 7 are the same as 7; that is, 

Tol — T and I o 7 = 7 ^ 3 ) 

♦ 


We conclude this section by noting that compositions can be defined for more than two linear transformations. For example, 
if 

Ty.U - *y, T 2 -.y — > W, and T 3 :W - *¥ 

are linear transformations, then the composition T 3 o 7 2 o T\ is defined by 

(7 3 o 7 2 o 7i) (n) = 7 3 (7 2 (7 t (u))) rzn 


(Figure 8.1.6). 


•- l\ • hu) 
















Figure 8.1.6 


The composition of three linear transformations. 


Exercise Set 8.1 




Click here for Just Ask! 


Use the definition of a linear operator that was given in this section to show that the function f-g} 
1- formula T(x h x 2 ) = (ii + 2x 2 , 3*i - x 2 ) is a linear operator. 


Use the definition of a linear transformation given in this section to show that the function RR 2 
2 ’ formula T(x h x 2 , * 3 ) = (2x i - x 2 4 - x 2 , x 2 - 4x 3 ) is a linear transformation. 

In Exercises 3-10 determine whether the function is a linear transformation. Justify your answer. 

7 : V —> A where V is an inner product space, and 7(u) = ||u||- 


T\F? _* 1 ? 3 , where vg is a fixed vector in R-' and 7(11) — u x vq- 


5. 


7 : m 22 —> M 22 , where B is a fixed 2x3 matrix and T(A) — AB- 


7: M nn -► R, where T(A) =tr(A)- 


F:M mn —* M nm , where F(A)=A 1 


8 . 


T:M 22 —> R where 


(a) 


£ 3 ! b 

c d 


= 3a — 4b 4 c — d 


(b) 


T 


a b 
c d 


= a 2 ±b 2 


T-.P 2 —* P-p where 


(a) T{a$4-a\x 4- a 2 x 2 ) = ag -I- a 1 (x + 1) + ^2 (x + 1)' 


^ p2 given by the 


R 2 given by the 


9 . 

















(b) T{a o 4 a\x ^ a 2 x 2 ) = («o + 1)4- Oi 4 l)x 4 (a 2 + l)* 2 


oo, oo) - >F(— oo , do ), where 

(a) T(f (x)) = l + f (x) 


(b) T(/(*))=/ (*4 1) 


Show that the function T in Example 9 is a linear operator. 

11 . 

Consider the basis £= (vi, v 2 ) for p}, where Vl = (1, 1) and y 2 = (1, 0), and let T.P? _* P? be the linear operator 

such that 

T(vi) = (l,- 2) and T(y 7 ) = (- 4,1) 

Find a formula for ^ 2 ), and use that formula to find T{5, — 3)- 

Consider the basis S— {vi, V2} f° r i? 2 , where Vl — ( _ 2, 1) and V2 — (1, 3), and let T.P? _* P? be the linear 

13* transformation such that 

T(vi) = ( - 1, 2, 0) and T(v 2 ) = (0, - 3, 5) 

Find a formula for T(x\, x 2 ), and use that formula to find T(2, — 3). 

Consider the basis S= {vi , v 2 , v 3 } for R 3 , where Vl = (1, 1, 1 ), v 2 = (1, 1, 0) , and V3 = ( 1, 0, 0) , and let 
14- t.P? _* j? 3 be the linear operator such that 

T(vi) = (2, - 1, 4), T(y 2 ) = (3, 0, 1), 7Xy 3 ) = ( - 1, 5, 1) 

Find a formula for T(x\, x 2 , x 3 )> and use that formula to find T{2,4, — 1). 

Consider the basis s= { Vl , v 2 , v 3 } for R 3 , where vi = (1, 2, 1). v 2 = (2, 9, 0), and V3 - (3, 3, 4), and let 
T:P? _i P? be the linear transformation such that 

T(y{) = (1, 0), T(y 7 ) = (- 1, 1), 7(v 3 ) = (0, 1) 

Find a formula for T(x\, x 2 , x 3 ), and use that formula to find T{1, 13, 7). 

Let vi, v 2 , and v 3 be vectors in a vector space V, and let r : y _, p? be a linear transformation for which 

16 ' TCvi7=Cl, -1,2 ), T(v 2 ) = ( 0,3,2), 7(v 3 ) = (-3,1,2) 

Find T{2\\ — 3v 2 H 4v 3 )- 


17 . 


Find the domain and codomain of T 2 o 7\> an d find (T 2 o T\)(x, > J ) 


(a) T\(x,y) = (2x,3y),T 2 (x,y) = (x-y,x I y) 


(b) T\(x,y) = (x - 3 y, 0), T 2 (x,y) = (4x - 5y, 3x - 6y ) 



(c) T\{x,y) = (2x, - 3y, x \ y), T 2 (x, y,z) = (x-y, yz) 


(d) T\(x,y) = (x-y,y + z,x-z),T 2 (x,y,z) = (0,x^y \ z) 

Find the domain and codomain of p 3 o T 2 q Ty and find (T 2 o T 2 oT\){x,y)- 

(a) T\(x,y) = ( - 2 y, 3x, x - 2y), T 2 (x,y,z) = (y,z, x), T 2 (x,y,z ) = (x 4 z,y-z) 

(b) T\{x,y) = (x +y,y, -x), T 2 (x,y,z ) = (0, x +y I z, 3y), T 2 (x,y,z ) = (3x + 2y,4z-x- 3y) 

Let T\ :M 22 -> R and T 2 \M 22 -► M 22 be the linear transformations given by T\ (A) — tr(A) and T 2 (A) —A T - 

(a) Find (ji o T 2 ")(A), where A — a b . 

(b) Can you find (T 2 a T\){A) rf - Explain. 


Let 7j -p n —* p n and T 2 \P n —* P n be the linear operators given by T\(p(x)) = p(x - 1) and T 2 (p(x)) = p(x + 1)- 
20 - Find (Tt c TiHpM) and (T 2 o T,)OW)- 

Let T\ : V —* V be the dilation T\ (v) = 4v- Find a linear operator T 2 :V —> V such that T\oT 2 = I and T 2 oT\= I- 


Suppose that the linear transformations T\\P 2 _*■ P 2 and T 2 :P 2 _> P 2 are given by the formulas 

22 ' T’iCpOO) — P(x 4-1) and T 2 (p(x)) -xp(x)- Find (7 2 o T\)(aQ 4- a\x 4 a 2 x 2 )- 

Let q^(x') be a fixed polynomial of degree m, and define a function T with domain p n by the formula 

23, row) =*(«(*))• 

(a) Show that T is a linear transformation. 

(b) What is the codomain of 77 


Use the definition of T 2 oT 2 o 7\ given by Formula 4 to prove that 

24 . 


(a) T 2 a T 2 a T\ is a linear transformation 


(b) T 2 o T 2 o T\ = (T 2 o T 2 ) o T\ 




(c) T3 o T 2 O T\ — T 2 o (T 2 o T\) 


Let TP? _i P? be the orthogonal projection of py onto the xy-plane. Show that ToT = 7- 

25. 


26. 

(a) Let T\ V —► If be a linear transformation, and let k be a scalar. Define the function (kT) : V —> Jf by 
(kT) (v) = k(T(y))- Show that kT is a linear transformation. 


(b) Find (3T)(x u x 2 ) ^T\R 2 -» p 2 is given by the formula T(x\, x 2 ) = (2*i -jr 2 , *2 4 *i> 


27. 

(a) Let Tj : _> ff and TV^_► ff be linear transformations. Define the functions (T\ I 7 2 ) '■ V —* ff and 

(Ti-T 2 ):V —* ffby 

{T\ 4 T 2 ) (v) = 7\ (v) f T 2 (v) 

(Ti-7 2 )(v) = r 1 (v)-r 2 (v) 

Show that T\ 4 T 2 and 7 ^ - T 2 are linear transformations. 

(b) Find (7^ 4 T 2 ) (x, y) and (J\ - T 2 ) (x, y) if 7\ \P 2 —* P 2 and T 2 \R 2 —* P 2 are given by the formulas 
7\{x,y) = (2 y, 3x) and T 2 (x,y) = (y,x)- 


(a) Prove that if a\,a 2 , by and k 2 are any scalars, then the formula 

F{x,y) = {a\x-\ b\y,a 2 x I b^y) 

defines a linear operator on p 2 . 

(b) Does the formula p( x , y) — (a\x 2 f b\y 2 , a 2 x 2 I b-ry 2 ") define a linear operator on p 2 l Explain. 


Let {vi, v 2 ,v H ) be a basis for a vector space V, and let T: V _> If be a linear transformation. Show that if 

29- ty V i ) = T(v 2 ) = ■■■ = T{\' n ) = 0. then T is the zero transformation. 


Let {vi, v 2 ,..., v H ) be a basis for a vector space V, and let T: V —» V be a linear operator. Show that if T(y\) = v 1? 
30. T(v 2 ) = v 2 ,. •T(v„) = v H , then T is the identity transformation on V. 


31. (For Readers Who Have Studied Calculus) Let 

D(f)=f f (x) and J{{)= 

J 0 





be the linear transformations in Examples Example 11 and Example 12. Find (JoD)(f) for 

(a) f (x) = x 2 4 3x 4 2 

(b) f(x) = sinx 

(c) f (x) — s 3 


32. (For Readers Who Have Studied Calculus) Let V = C[a, b] be the vector space of functions continuous on [a,b], 
and let T : V —► V be the transformation defined by 

T(f)=5f(x)+3j X f(t)dt 

Ja 

Is T a linear operator? 


Discussion 

Discovery 


Indicate whether each statement is always true or sometimes false. Justify your answer by 
33. giving a logical argument or a counterexample. In each part, V and W are vector spaces. 


(a) If r( C ivi -l- c 2\'2) = c\7(x\) 4 c 2 T(y 2 ) for a11 vectors vi and v 2 in V and all scalars 
c\ and c 2 , then Lis a linear transformation. 

(b) If v is a nonzero vector in V, then there is exactly one linear transformation 
T:V —*W such that T( - v ) = - T(y ). 

(c) There is exactly one linear transformation T : y —> W for which 7Y U |. v ) = T(u — v) 
for all vectors u and v in V. 

(d) If vq is a nonzero vector in V, then the formula T(v) = vq 4 v defines a linear operator 
on V. 


If B — (vi, \’2,..., v„} is a basis for a vector space V, how many different linear operators can 
34. b e created that map each vector in B back into B ! Explain your reasoning. 

Refer to Section 4.4. Are the transformations from p n to p m that correspond to linear 
35* transformations from to necessarily linear transformations from p n to p m ? 
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8.2 

KERNEL AND RANGE 


In this section we shall develop some basic properties of linear 
transformations that generalize properties of matrix transformations obtained 
earlier in the text. 


Kernel and Range 

Recall that if A is an m x n matrix, then the nullspace of A consists of all vectors x in R n such that Ax = 0, and by Theorem 5.5.1 
the column space of A consists of all vectors b in R m for which there is at least one vector x in R n such that Ax = b- From the 
viewpoint of matrix transformations, the nullspace of A consists of all vectors in R 71 that multiplication by A maps into 0, and 
the column space of A consists of all vectors in R m that are images of at least one vector in R 71 under multiplication by A. The 
following definition extends these ideas to general linear transformations. 


DEFINITION 


If T. V —> W >s a linear transformation, then the set of vectors in V that T maps into 0 is called the kernel of T; it is denoted 
by ker(7). The set of all vectors in W that are images under T of at least one vector in V is called the range of T; it is denoted 
by R(T)- 


Kernel and Range of a Matrix Transformation 


If Tj^.R 71 — * R m is multiplication by the m x n matrix A, then from the discussion preceding the definition above, the kernel of 
is the nullspace of A, and the range of R A is the column space of A. 

♦ 


Kernel and Range of the Zero Transformation 


Let T; V —> W be the zero transformation (Example 2 of Section 8.1). Since T maps every vector in V into 0, it follows that 
ker(F) = V- Moreover, since 0 is the only image under T of vectors in V, we have R(T) — {0} • 

♦ 


Kernel and Range of the Identity Operator 


Let /; y —> y be the identity operator (Example 3 of Section 8.1). Since /( v ) = v for all vectors in V, every vector in V is the 
image of some vector (namely, itself); thus ^(/) = y. Since the only vector that I maps into 0 is 0, it follows that ker(/) = {0} • 



















Kernel and Range of an Orthogonal Projection 


Let T.B? _» R? be the orthogonal projection on the xy-plane. The kernel of T is the set of points that T maps into 0 = (0, 0, 0) 

; these are the points on the z-axis (Figure 8.2.1a). Since T maps every point in RR into the xy-plane, the range of T must be 
some subset of this plane. But every point (xq, yg, 0) in the *y-plane is the image under T of some point; in fact, it is the image 
of all points on the vertical line that passes through (^ 0i 0) (Figure 8.2. lb). Thus R(T) is the entire ^y-plane. 




Figure 8.2.1 

(a) ker(T) is the z-axis. ( b) R(T) is the entire ^y-plane. 


EXAMF Kernel and Range of a Rotation 


Let T\R? _> R? be the linear operator that rotates each vector in the xy-plane through the angle Q. (Figure 8.2.2). Since every 

"i 

vector in the xjy-plane can be obtained by rotating some vector through the angle 0. (why?), we have R(T) = R ■ Moreover, the 
only vector that rotates into 0 is 0 , so ker(T) = (0) • 



Calculus RequireHernel of a Differentiation Transformation 
















Let V = C 1 ( — do , do ) be the vector space of functions with continuous first derivatives on ( ^ oo , do ), let 
W = F{ — oo, oo)be the vector space of all real-valued functions defined on ( _ oo , do ) , and let l)\ V — > W be the 
differentiation transformation D(f) = j ! (x). The kernel of D is the set of functions in V with derivative zero. From calculus, 
this is the set of constant functions on ( _ oo , oo )• 


Properties of Kernel and Range 

In all of the preceding examples, ker(T) and R(T) turned out to be subspaces. In Examples Example 2, Example 3, and 
Example 5 they were either the zero subspace or the entire vector space. In Example 4 the kernel was a line through the origin, 
and the range was a plane through the origin, both of which are subspaces of pf. All of this is not accidental; it is a consequence 
of the following general result. 


THEOREM 8.2.1 


If T V —» W is linear transformation, then 
(a) The kernel of T is a subspace of V. 


(b) The range of T is a subspace of W. 


To Show that ker(T) is a subspace, we must show that it contains at least one vector and is closed under addition and 
scalar multiplication. By part (a) of Theorem 8.1.1, the vector 0 is in ker(7), so this set contains at least one vector. Let \q and 
\'2 be vectors in ker(7> and let k be any scalar. Then 


SO VI +v 2 is in ker(T). Also, 


SO tvj is in ker(T)- 


T(v i -I- v-/) = T{\ i) + T(yf) = 0 4 0 = 0 


T(kv i) =£T(vi) =£0 = 0 


Proof (b) Since T(0) = 0- there is at least one vector in R(T)- Let wj and W2 be vectors in the range of T, and let k be any 
scalar. To prove this part, we must show that Wl | W9 and jtwj are in the range of T; that is, we must find vectors a and lj in V 
such that T(a) = wj -|- W2 an d T(b) = jhvj. 


Since wj and W2 are in the range of T, there are vectors aj and a2 in V such that T{-n\) = wj and Tiyf) = w> Let a = ai 4- a2 
and b = Ara i • Then 

T(a) = 7(ai + = T(ai) 4- T( a^) =wi + W2 


and 


7(b) = T (.ta i) =kT( ai) =k\w { 


which completes the proof. 


In Section 5.6 we defined the rank of a matrix to be the dimension of its column (or row) space and the nullity to be the 
dimension of its nullspace. We now extend these definitions to general linear transformations. 






DEFINITION 


If T\ V —* W is a linear transformation, then the dimension of the range of T is called the rank of T and is denoted by 
rank ( 7 ); the dimension of the kernel is called the nullity ofT and is denoted by nullity(T)- 


If A is an m x n matrix and Tj(.R n —> R m is multiplication by A, then we know from Example 1 that the kernel of 7 ^ is the 
nullspace of A and the range of 7 ^ is the column space of A. Thus we have the following relationship between the rank and 
nullity of a matrix and the rank and nullity of the corresponding matrix transformation. 


THEOREM 8.2.2 


If A is an m x n matrix and Tj±.R n - * R m is multiplication by A, then 

(a) nullity (T a) = nullity (Jl) 

(b) rank(T a) = rank( 2 l) 


Finding Rank and Nullity 


Let Ta R^ _> j ? 4 be multiplication by 


Find the rank and nullity of 7 ^. 


-1204 5-3 

3-720 1 4 

2-524 6 1 

4 _92-4-47 


Solution 

In Example 1 of Section 5.6, we showed that rank (A) = 2 and nullity(71) = 4- Thus, from Theorem 8.2.2, we have rank(7\q) = 2 
and nuUityfT’ a) =4- 


Finding Rank and Nullity 


Let T\P? _* P? be the orthogonal projection on the xy-plane. From Example 4, the kernel of T is the z-axis, which is 

one-dimensional, and the range of T is the xy-plane, which is two-dimensional. Thus 

nullity(T) = 1 and rank(T) = 2 



















Dimension Theorem for Linear Transformations 


Recall from the Dimension Theorem for Matrices (Theorem 5.6.3) that if A is a matrix with n columns, then 

rank (A) 4- nullity (A) = n 

The following theorem, whose proof is deferred to the end of the section, extends this result to general linear transformations. 


THEOREM 8.2.3 

Dimension Theorem for Linear Transformations 

If T:V —> W is a linear transformation from an n-dimensional vector space V to a vector space W, then 

rank(T) + nuUity(T) = n ^ 

In words, this theorem states that for linear transformations the rank plus the nullity is equal to the dimension of the domain. 
This theorem is also known as the Rank Theorem. 


If A is an m x « matrix and T a R n —► R m is multiplication by A, then the domain of T A has dimension n, so Theorem 
8.2.3 agrees with Theorem 5.6.3 in this case. 


EXAMPLE 9 Using the Dimension Theorem 

Let 7\B? _> B? be the linear operator that rotates each vector in the xy-plane through an angle 9 . We showed in Example 5 

that ker(T) = {0} and R{T) = R ?. Thus 

rank(T) f nuUity(T) =2 + 0 = 2 

which is consistent with the fact that the domain of T is two-dimensional. 

Additional Proof 


Proof of Theorem 8.2.3 We must show that 


dim(3?(T)) + dim(ker(T)) = n 

We shall give the proof for the case where l <dim(ker(T)) <n. The cases where dim(ker(T)) = 0 and 
dan(ker(r)) = n are left as exercises. Assume dim(ker(7)) =r, and let vi,..., v,. be a basis for the kernel. 
Since {vi,..., \>) is linearly independent, Theorem 5.4.6b states that there are n — r vectors, \> + i, ..., v H 
, such that the extended set { V1 ,..., \>+i,..., v„) is a basis for V. To complete the proof, we shall 

show that the n -r vectors in the set s= {T(\> + i7(v„)} form a basis for the range of T. It will then 
follow that 

dim(i?(7^) f dim(ker(7^)) = (« — r) + r = n 

First we show that S spans the range of T. If lj is any vector in the range of T, then b = T’(v) for some vector v in V. Since 








{vj, v r , v,.+i,v„} is a basis for V, the vector v can be written in the form 

v = civi +“- + C r T r + C r+ l\>+l H-t- c»v„ 

Since vj,..., v r lie in the kernel of T, we have T(vi) = ■■■= T(y r ) = 0, so 

b = T(v) = c r+l T(x r+l ) + - + c n T(y„) 


Thus S spans the range of T. 

Finally, we show that S is a linearly independent set and consequently forms a basis for the range of T. Suppose that some linear 
combination of the vectors in S is zero; that is. 


*, + i7Xy, + i) F — I k„T(y„)= 0 


( 2 ) 


We must show that £j , +1 =... = k n = 0- Since T is linear, 2 can be rewritten as 

T(ky+\Xy+\ +- + jt M v„) =0 

which says that -|- -■ 4 . k n v n is in the kernel of T. This vector can therefore be written as a linear combination of the 

basis vectors (vj, v }, say 

*I-+1T|-+1 +.+ = *1V1 H-h k r v r 


Thus, 
Since ( Vli 


jfci VI +- + jt,.v,-jt, + iv, + i --£„V M = 0 

v H ) is linearly independent, all of the k's are zero; in particular, = ■■■ = k n = 0 - which completes the proof. 


Exercise Set 8.2 



Click here for Just Ask! 


Let T.R? _> R l be the linear operator given by the formula 

T{x,y) = ( 2 x-y, - 8 * + Ay) 

Which of the following vectors are in R(T)1 

(a) (1. -4) 

(b) (5,0) 

(c) (-3,12) 

Let T.R? _* R? be the linear operator in Exercise 1. Which of the following vectors are in ker(T) 

(a) (5,10) 


(b) (3,2) 







(C) (1.1) 


Let T' R 4 _> P? be the linear transformation given by the formula 

3. 

T(x\, X2, *3, X4) = (4*1 ^ X2 — 2x2 ~ 3*4, 2 x^ *2 f X3 — 4x4, 6*1 ~ 9*3 I 9*4) 

Which of the following are in R{T)1 

(a) (0,0,6) 

(b) (1,3,0) 

(c) (2,4,1) 

Let T:R 4 _> P? be the linear transformation in Exercise 3. Which of the following are in ker(T)? 

4. 

(a) (3, - 8 , 2, 0) 

(b) ( 0 , 0 , 0 , 1 ) 

(c) (0, -4, 1,0) 

Let T.Pj _* P 2 b e the linear transformation defined by T(p(x)) = xp(x)- Which of the following are in ker(T)? 

5. 

(a) x 2 

(b) 0 

(c) 1 +* 

Let T.P'i _* P 3 be the linear transformation in Exercise 5. Which of the following are in RXT'p- 

6 . 

(a) x f x 2 

(b) 1 I * 

(c) 3 — x 2 





Find a basis for the kernel of 

7 . 

(a) the linear operator in Exercise 1 


(b) the linear transformation in Exercise 3 


(c) the linear transformation in Exercise 5. 


Find a basis for the range of 

8 . 

(a) the linear operator in Exercise 1 


(b) the linear transformation in Exercise 3 


(c) the linear transformation in Exercise 5. 


Verify Formula 1 of the dimension theorem for 

9. 

(a) the linear operator in Exercise 1 

(b) the linear transformation in Exercise 3 

(c) the linear transformation in Exercise 5. 

In Exercises 10-13 let Tbe multiplication by the matrix A. Find 

(a) a basis for the range of T 

(b) a basis for the kernel of T 


(c) the rank and nullity of T 


(d) the rank and nullity of A 



11 . 



0 - 1 
0 -2 

0 0 


1 5 2 

2 3 0 


1 4 5 0 9 

3-2 10-1 

1 0 - 10-1 

2 3 5 1 8 


Describe the kernel and range of 

14 . 

(a) the orthogonal projection on the x^-plane 


(b) the orthogonal projection on the yz-plane 


(c) the orthogonal projection on the plane defined by the equation y = x 


Let V be any vector space, and let T\ V —* V be defined by T(y") = 3v- 

(a) What is the kernel of 77 


(b) What is the range of 77 


In each part, use the given information to find the nullity of T. 

16 . 

(a) t.R 5 _» R 7 has rank 3. 


(b) T P 4 _» P '1 has rank 1 . 


(c) The range of p.R 6 _* 7? 3 is i? 3 . 


(d) T: M'i'i _* Mji has rank 3. 

Let A be a 7 x 6 matrix such that Ax = 0 has only the trivial solution, and let r : R 6 _, R 7 be multiplication by A. Find the 

rank and nullity of T. 


A = 


12. A = 


13 . 


A = 


18 . 


Let A be a 5 x 7 matrix with rank 4. 


(a) What is the dimension of the solution space of j4x = 0? 


(b) Is j4]t = b consistent for all vectors b in p 5 7 Explain. 

Let T\B? _* V be a linear transformation from p} to any vector space. Show that the kernel of T is a line through the 

19* origin, a plane through the origin, the origin only, or all of p?. 

Let 7 : y _, p} be a linear transformation from any vector space to p}. Show that the range of T is a line through the 

20. or igi nj a plane through the origin, the origin only, or all of p?. 

Let T P? _» F? be multiplication by 

21 . 


(a) Show that the kernel of T is a line through the origin, and find parametric equations for it. 

(b) Show that the range of T is a plane through the origin, and find an equation for it. 

Prove: If { V i, \'2, ■■ } is a basis for V and wj, W 2 , •••, w H are vectors in W, not necessarily distinct, then there exists a 

22* linear transformation T. V —» W such that 

T(v i) = wi, T(vj) = W},.... T(v„) = w„ 

Lor the positive integer n > ], let T : M ym —> R be the linear transformation defined by T(A) = tr(j4)> for A an n x n matrix 
23* with real entries. Determine the dimension of ker(T)- 

Prove the dimension theorem in the cases 

24. 

(a) dim(ker(T)) = 0 

(b) dim(ker(T)) = n 


25. (For Readers Who Have Studied Calculus) Let D.P-$ —> P-j be the differentiation transformation D(p) = p ! (y ). 
Describe the kernel of D. 


(For Readers Who Have Studied Calculus) Let j-p^ _* R be the integration transformation _ 


/- 


p(x)dx- 


1 3 4 

3 4 7 

-2 2 0 


Describe the kernel of J. 



27. (For Readers Who Have Studied Calculus) Let £): V —* W be the differentiation transformation D(f ') = /'(*), where 
V = C 3 (— oo, oo) and W = F ( — oo , oo ) • Describe the kernels of DoD and Do DoD 


Discussion 

Discovery 


Fill in the blanks. 

28. 


(a) If Tj^.R n —* R m is multiplication by A, then the nullspace of A corresponds to the 

_of T a , and the column space of A corresponds to the_of p A 

(b) If T\R? _ * R? is the orthogonal projection on the plane x | y | z — 0, then the kernel of 

T is the line through the origin that is parallel to the vector_. 

(c) If V is a finite-dimensional vector space and T: V —» W is a linear transformation, then 

the dimension of the range of T plus the dimension of the kernel of T is_. 

(d) If Tj^.R? _> R? is multiplication by A, and if rank(T^) = 2, then the general solution of 

Ax = 0 has_(howman y?) parameters. 


29. 

(a) If T\R? _> R? is a linear operator, and if the kernel of Lis a line through the origin, then 

what kind of geometric object is the range of 77 Explain your reasoning. 

(b) If T\R? _> R? is a linear operator, and if the range of Lis a plane through the origin, then 

what kind of geometric object is the kernel of 77 Explain your reasoning. 


30. (For Readers Who Have Studied Calculus) Let Vbe the vector space of real-valued functions 
with continuous derivatives of all orders on the interval ( _ oo , oo ), and let W = F( — oo , oo ) 
be the vector space of real-valued functions defined on ( _ qq ; oo )• 

(a) Find a linear transformation T~. V —> W whose kernel is Py 

(b) Find a linear transformation T\ V —> W whose kernel is p . 


If A is an m x n matrix, and if the linear system Ax = b is consistent for every vector lj in R: n , 
31* what can you say about the range of Tj^.RJ 1 —* R m ^ 
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8.3 

INVERSE LINEAR 
TRANSFORMATIONS 


In Section 4.3 we discussed properties of one-to-one linear transformations 
from R n to R m . In this section we shall extend those ideas to more general 
kinds of linear transformations. 


Recall from Section 4.3 that a linear transformation from R n to R m is called one-to-one if it maps distinct vectors in R n into 
distinct vectors in R m . The following definition generalizes that idea. 


DEFINITION 


A linear transformation T: V -► W is said to be one-to-one if T maps distinct vectors in V into distinct vectors in W. 


A One-to-One Linear Transformation 


Recall from Theorem 4.3.1 that if A is an n x n matrix and Tj^.R n -► R. n is multiplication by A, then R^ is one-to-one if 

and only if A is an invertible matrix. 

♦ 


A One-to-One Linear Transformation 


Let T: P n —P H _, j be the linear transformation 

T{v) = T{p(x))=xp{x) 

discussed in Example 8 of Section 8.1. If 

p = p(x) — eg 4- c^x I-h c n x n and f\ — q{x)—d^Ad\x-\-—Ad yi x yi 

are distinct polynomials, then they differ in at least one coefficient. Thus, 

T(p) = cox 4- ci* 2 4-and T{(\) —d^x A d\x^ A -1- 

also differ in at least one coefficient. Thus T is one-to-one, since it maps distinct polynomials p and 1 into distinct 
polynomials and 

♦ 


A Transformation That Is Not One-to-One 


Calculus Required 



















Let 


D : C 1 ( — do , oo ) -► F( — do , do ) 

be the differentiation transformation discussed in Example 11 of Section 8.1. This linear transformation is not one-to-one 
because it maps functions that differ by a constant into the same function. For example, 

D{x 2 ) = D(x 2 f 1) = 2x 

♦ 

The following theorem establishes a relationship between a one-to-one linear transformation and its kernel. 

THEOREM 8.3.1 

Equivalent Statements 

If T: V -► W is a linear transformation, then the following are equivalent. 

(a) T is one-to-one. 

(b) The kernel ofT contains only the zero vector; that is, ker(T) = {0} • 

(c) nullity(T) — 0- 


The equivalence of ( b ) and (c) is immediate from the definition of nullity. We shall complete the proof by proving 
the equivalence of (a) and ( b ). 

Assume that T is one-to-one, and let v be any vector in ker(T)- Since v and 0 both lie in ker(7)> we have 7(y) = 0 
and 7(0) = 0, so 7(y) = T( 0)- But this implies that v = Q, since T is one-to-one; thus ker(7j contains only the zero vector. 

Assume that ker(T) = {0} and that v and w are distinct vectors in V; that is, 


To prove that T is one-to-one, we must show that T(y) and T(y) are distinct vectors. But if this were not so, then we would 
have T(y) — T(y) ■ Therefore, 

T (v) - T (w) =0 or T{\ - w) = 0 

and so v _ w is in the kernel of T. Since ker(F) = {0}, this implies that v _ w = 0, which contradicts 1. Thus T(y) and 
T’(^v) must be distinct. 


Using Theorem 8.3.1 


In each part, determine whether the linear transformation is one-to-one by finding the kernel or the nullity and applying 
Theorem 8.3.1. 







(a) T.R 2 _► R 2 rotates each vector through the angle &. 


(b) y R 2 _» R? is the orthogonal projection on the xy-plane. 

(c) t.R 6 _> R 4 is multiplication by the matrix 


-1 20 4 5-3 

3 -7 2 0 1 4 

2-52 4 6 1 

4-9 2-4-4 7 


Solution (a) 

From Example 5 of Section 8.2, kerf 7") = {0 }, so 7" is one-to-one. 

Solution (b) 

From Example 4 of Section 8.2, ker(T) contains nonzero vectors, so T is not one-to-one. 

Solution (c) 

From Example 7 of Section 8.2, nullity(T) = 4, so T is not one-to-one. 

+ 

In the special case where T is a linear operator on a. finite-dimensional vector space, a fourth equivalent statement can be 
added to those in Theorem 8.3.1. 


THEOREM 8.3.2 

IfV is a finite-dimensional vector space, and T\ V -> V is a linear operator, then the following are equivalent. 

(a) T is one-to-one. 

(b) ker(7^= {0}- 

(c) nullity(T) — 0- 

(d) The range ofT is V; that is, R{T) — V- 









We already know that (a), ( b ), and (c) are equivalent, so we can complete the proof by proving the equivalence of (c) 

and (cl). 

Suppose that dirnff'') = n and nullity(7) = 0- It follows from the Dimension Theorem (Theorem 8.2.3) that 

rank(T) = n — nuUity(T) = n 

By definition, rank(T) is the dimension of the range of T, so the range of T has dimension n. It now follows from Theorem 
5.4.7 that the range of T is V , since the two spaces have the same dimension. 

Suppose that dirnff'') = n and R(T) = V . It follows from these relationships that dim(3?(7)) = n, or, equivalently, 
rank(7") = n- Thus it follows from the Dimension Theorem (Theorem 8.2.3) that 

nuUitytT) = n — rank(T) = n — n = 0 


A Transformation That Is Not One-To-One 


Let T A .R 4 _* R^ be multiplication by 


Determine whether r a is one-to-one. 


A = 


1 1 


-2 4 
-4 8 
1 5 

4 8 


Solution 

As noted in Example 1, the given problem is equivalent to determining whether A is invertible. But det(A) = 0, since the 
first two rows of A are proportional, and consequently, A is not invertible. Thus R A is not one-to-one. 

♦ 


Inverse Linear Transformations 

In Section 4.3 we defined the inverse of a one-to-one matrix operator T A .R n —+ R 71 to be T^_i \R n —► R n , and we 
showed that if w is the image of a vector x under t a then T A -\ = maps w back into x . We shall now extend these ideas 

to general linear transformations. 


Recall that if T : y —> W is a linear transformation, then the range of T, denoted by R(T), is the subspace of W consisting of 
all images under T of vectors in V. If T is one-to-one, then each vector w in R( T) is the image of a unique vector v in V. This 
uniqueness allows us to define a new function, called the inverse ofT and denoted by 7 -1 , that maps w back into v (Figure 
8.3.1). 


v 



n- = 7'(v i 
R(T) 


Figure 8.3.1 


The inverse of T maps T(\) back into v . 










V is a linear transformation. Moreover, it follows from the definition of 


It can be proved (Exercise 19) that T 1 .R{T) 
T~ x that 


r“ 1 (J’(v))=7- 1 (w)=v 

(2a) 

T(7- 1 (w))=7(v)=w 

(2b) 


so that T and 7 -1 , when applied in succession in either order, cancel the effect of one another. 


Remark It is important to note that if R- V —* W is a one-to-one linear transformation, then the domain of y -1 is the 
range of T. The range may or may not be all of W. However, in the special case where T: V —* V is a one-to-one linear 
operator, it follows from Theorem 8.3.2 that R{T) — V\ that is, the domain of T~ X is all of V. 


An Inverse Transformation 


In Example 2 we showed that the linear transformation T: P n —* P n ^ 1 given by 

T( V ) = T{p{x))=xp{x) 

is one-to-one; thus, T has an inverse. Here the range of T is not all of 7 H+1 ; rather, EXT) is the subspace of + 1 consisting 
of polynomials with a zero constant term. This is evident from the formula for T\ 

T(c 0 -I- Cl* I- hC„I H ) = CQX 4- C\x 2 4- hc H / +1 

It follows that T~ X R(T) _+ P„ i s given by the formula 

T _1 (cox 4- cix 2 4- - 4- c„x H+1 ) = co 4- cix 4- - 4- c„x M 
For example, in the case where n > 3, 

T~ X (2x - x 2 + 5x 3 + 3x 4 ) = 2 - * 4 - 5x 2 4 - 3x 3 


An Inverse Transformation 


Let T.R? _► R 3 be the linear operator defined by the formula 

T(x 1 , x% x^) = (3xi + *i, - 2xi - 4x^ -I- 3x^, 5xi + 4x^ - 2x^) 
Determine whether T is one-to-one; if so, find 7 ,_1 (^ 1 X j m x^)- 


Solution 


From Theorem 4.3.3, the standard matrix for T is 


[T] 


3 1 0 

-2-4 3 

5 4-2 


(verify). This matrix is invertible, and from Formula 1 of Section 4.3, the standard matrix for 7 1 is 














It follows that 


[T~ l ] = [T ]- 1 


4 -2 -3 
-11 6 9 

-12 7 10 


pr 

= [r _1 ] 

■*r 


4 

-2 

-3' 

■*r 


4xi - 2x2 - 

3x 3 

*2 

*2 

= 

-11 

6 

9 

*2 

= 

— 1 lx 1 4 6x2 4 

9x3 

'J/3 

j 

*3 


-12 

7 

10 



—12xi+7x2+ 

10x3 


Expressing this result in horizontal notation yields 

7 _1 (x i, X2, xf) = (4xi - 2 x 2 - 3 x 3 , - 11*1 4 6 x 2 4- 9 x 3 , - 12xi 4 7 x 2 4 IOX 3 ) 


The following theorem shows that a composition of one-to-one linear transformations is one-to-one, and it relates the inverse 
of the composition to the inverses of the individual linear transformations. 


THEOREM 8.3.3 


If T\\U _* V an d T-r.V _► W are one-to-one linear transformations, then 

( a ) Ti o T\ I s one-to-one. 

(b) (7 2 o7i) _1 = 7f 1 o7 2 _1 . 


)f (a) We want to show that Tj o T\ maps distinct vectors in U into distinct vectors in W. But if u and v are distinct 
vectors in U, then T\ ( 11 ) and T\ (v) are distinct vectors in V since f\ is one-to-one. This and the fact that 7 ^ is one-to-one 
imply that 


72(7! (u)) and 7 2 (7 i(v)) 

are also distinct vectors. But these expressions can also be written as 

(7 2 o7i)(u) and (7 2 o7i)(v) 

so 7 2 ci 7i maps u and v into distinct vectors in W. 


Proof (b) We want to show that 


(Tl o ro -■ (w) = (Tf 1 o 37 1 ) (w) 

for every vector w in the range of 7 2 o 7p For this purpose, let 

«=(7 2 o7 1 )- 1 (w) 


( 3 ) 


u=(7f 1 o7 2 - 1 )(w) 


so our goal is to show that 

















But it follows from 3 that 


(T 2 o Ti)(u) =w 


or, equivalently, 


7 2 (ri(u))=w 


Now, taking ^-1 of each side of this equation and then ^-l of each side of the result and using 2a 
yields (verify) 

u=Tf‘ (r 2 -‘(w» 

or, equivalently, 

n=(77 1 o77 1 )(w) 


In words, part (b) of Theorem 8.3.3 states that the inverse of a composition is the composition of the inverses in the reverse 
order. This result can be extended to compositions of three or more linear transformations; for example, 

(73 - 7-2 O 7-,) - 1 = Tf 1 O T? a T? (4) 

In the case where p A , T B , and are matrix operators on pf\ Formula 4 can be written as 

/ *~p ■'~r ■'~r \ —1 ■T ? —1 ■T ? —1 ■T ? —1 

{l c® l £ A) = l A oI B 0 1 C 

which we might also write as 

(Tcba)- 1 = T a -i b -i c -i ( 5 ) 

In words, this formula states that the standard matrix for the inverse of a composition is the product of the inverses of the 
standard matrices of the individual operators in the reverse order. 

Some problems that use Formula 5 are given in the exercises. 

Dimension of Domain and Codomain 

In Exercise 16 you are asked to show the important fact that if V and W are finite-dimensional vector spaces with 
dim(Jf) < and if T: V — » W is a linear transformation, then T cannot be one-to-one. In other words, the dimension 

of the codomain W must be at least as large as the dimension of the domain V for there to be a one-to-one linear 
transformation from V to W. This means, for example, that there can be no one-to-one linear transformation from space R 3 to 
the plane pf. 


Dimension and One-to-One Linear Transformations 


A linear transformation 7 ^ from the plane 7 2 to the real line R has a standard matrix 

A— [a b] 


If u = (x,y) is a point in p 2 , its image is 


T^iCu) =ax I -by 

which is a scalar. But if ax + by — k, say, then there are infinitely many other points v in p} that also have 7 f.y) — k , since 
there are infinitely many points on the line 


ax 4- by = k 

This is because if a and b are nonzero, then every point of the form 





V = 


has T a (\) — k , whereas if a — 0 but b is nonzero, then every point of the form 

v= (x, 0) 

has T a (v) — 0, and if b — 0 but a is nonzero, then every point of the form 

v= (0 ,y) 

has T a (v) = 0- Finally, in the degenerate case a = 0 and h = 0> we have T A {\) = 0 for every v in pA 

In each case, T fails to be one-to-one, so there can be no transformation from the plane to the real line that is both linear and 
one-to-one. 

♦ 


Of course, even if dim (IF) > dirn(F), a linear transformation from V to W might not be one-to-one, as the zero 
transformation shows. 


Exercise Set 8.3 

Click here for Just Ask! 

In each part, find ker(T). and determine whether the linear transformation T is one-to-one. 

(a) T:R 2 —> R 2 , where T{x, y) = (y, x) 

(b) t-.R 2 —» R 2 , where T(x, y) = (0, 2* + 3y) 

(c) T: R 2 —> R 2 , where T(x, y) = (x + y, x - y) 

(d) T:R 2 —> where T(x, y) = (x, y,x + y) 

(e) t-.R 2 —^ R 3 , where T(x, y) = (x -y, y - x, 2x - 2y) 

(f) 7: R 3 — >R 2 , where T(x,y,z) = (x + y-\-z,x-y-z) 

In each part, let T.R 2 _> R 2 be multiplication by A. Determine whether T has an inverse; if so, find 



(a) 


A = 


5 2 
2 1 

















(b) 



-3 

-2 


(c) 


A = 


4 7 

-1 3 


In each part, let TP? 

3. 


p?-' be multiplication by A. Determine whether T has an inverse; if so, find 


( 

'*r 

\ 


x 2 


L 

x 3 



(a) 152 

A= 1 2 1 

-1 1 0 


(b) 14-1 

A— 12 1 

-1 1 0 


(c) 101 

A— Oil 
1 1 0 


(d) 1-11 

A= 0 2 -1 

2 3 0 


In each part, determine whether multiplication by A is a one-to-one linear transformation. 

4. 


(a) 



-2 

-4 

6 


(b) 13 5 7 

A= 2 -124 

-13 0 0 


(c) 



-2 

5 

3 



As indicated in the accompanying figure, let X'R? _> [;} be the orthogonal projection on the line y — x. 

5 . 


(a) Find the kernel of T. 


(b) Is T one-to-one? Justify your conclusion. 



As indicated in the accompanying figure, let x.R 2 _ * R? be the linear operator that reflects each point about the y-axis. 

6 . 

(a) Find the kernel of T. 

(b) Is T one-to-one? Justify your conclusion. 


i 1 


/'m «~ 




A 


Figure Ex-6 


In each part, use the given information to determine whether the linear transformation T is one-to-one. 

7. 

(a) T\R™ -> R m \ nulkty(7^ = 0 


(b) TR n —* R H ; rank(T) = n - 1 

(c) T\R m - *R n ;n<m 

(d) T:R” — >R”;R(T) =R” 


8 . 


In each part, determine whether the linear transformation T is one-to-one. 













(a) T: P 2 —» Py where 7 ’(a n + 4 - a^ 2 ) = x(an 4- aix 4- fl;>x 2 ) 


(b) T:P' 2 —* Py w ^ ere T{p{x)) = p(x 4 1) 


Let A be a square matrix such that det(A) = 0- Is multiplication by A a one-to-one linear transformation? Justify your 
9* conclusion. 

In each part, determine whether the linear operator p-R n _» R n is one-to-one; if so, find T~ l (x\, x 2 ,x n )- 

10 * 

(a) T(x 1 , x 2 ,x„) = ( 0 , x\, x 2 , x H _i) 

(b) T(x 1 , x 2 ,x„) = (x„, x„_i, x 2 , xi) 

(c) T(x 1 , x 2 ,x„) = (x 2 , X 3 ,x„, xi) 

Let T.R n —► R n be the linear operator defined by the formula 
n * T(x\,x%..., x„) - (a\x\,a 2 x 2 ,...,a yi xy i ) 

where a 1 , ..., a n are constants. 

(a) Under what conditions will T have an inverse? 

(b) Assuming that the conditions determined in part (a) are satisfied, find a formula for 7 -1 ( x 1> X 2 ,x n )• 

Let Ty.R? _» R? and T 2 .R? _> R? be the linear operators given by the formulas 

12 ' T\(x,y) = (xy,x-y) and T 2 {x,y) = ( 2 x + y, x — 2 y) 

(a) Show that T\ and T 2 are one-to-one. 


(b) Find formulas for 7-1 y) , and 7-1 y y and (J 2 o T x )~ { (x, y). 


(c) Verify that (J 2 o T \) _1 = Tf 1 o T 2 _1 • 


Let T\ \P 2 _> P3 and T 2 :P 2 _► P 2 be the linear transformations given by the formulas 

T\{p{x))=xp(x) and T 2 (p(x)) = p(x + 1) 


(a) Find formulas for 7-1 ( p ( x )), 7f 1 0?(x))> and <J 2 o T{) _1 (p(x))- 



(b) Verify that <J 2 0 T\)~ l = 7f 1 o T 2 _1 . 


Let Tj^.F? _> Tb'F? _► F?^ and Tq.F? _> F? be the reflections about the xy-plane, the ^-plane, and the yz 

-plane, respectively. Verify Formula 5 for these linear operators. 

Let T.P\ _* P? be the function defined by the formula 

15 ' T(p(x)) = (p(Q),p(\)) 


(a) Find TO-2*)- 

(b) Show that T is a linear transformation. 

(c) Show that T is one-to-one. 

(d) Find 7 7-1 (2, 3), and sketch its graph. 


Prove: If V and W are finite-dimensional vector spaces such that dim W < dim V, then there is no one-to-one linear 
16. transformation T: V —* W- 


17. 


In each part, determine whether the linear operator M 22 _+ M 22 ’ s one-to-one. If so, find 


’-1 


a b 
c d 


(a) 


a b 
c d 


a 0 
0 d 


(b) 


£ 3 ! b 

c d 


a c 
b d 


(c) 


a b 
c d 


d -b 

— c a 


Let T.P? _i F? be the linear operator given by the formula T(x, y) = (x +- ky, -y)- Show that T is one-to-one for 

18* every real value of k and that t ,_ 1 _ j-. 


Prove that if T\ V -► W is a one-to-one linear transformation, then T 1 ,F(T) -> V is a linear transformation. 


19 . 



(For Readers Who Have Studied Calculus) Let J:P\ —> R be the integration transformation 
Determine whether J is one-to-one. Justify your conclusion. 



p(x")dx- 


21. (For Readers Who Have Studied Calculus) Let Vbe the vector space C 1 [0, 1 ] and let T: V —► R be defined by 
T(f) — f (0) 4 - 2/ '(0) -I- 3f ! (\). Verify that Lis a linear transformation. Determine whether T is one-to-one. Justify 
your conclusion. 

In Exercises 22 and 23, determine whether 7 ^ 0 7 ^ = t 2 o T\ • 


22 . 

(a) T^ -R 2 _, R 1 is the orthogonal projection on the x-axis, and TpR? _► R? i s the orthogonal projection on the 

y-axis. 

(b) T\-.R? _ ► R? i s the rotation about the origin through an angle and R 2 .R l _* R? i s the rotation about the 

origin through an angle 9 2 - 

(e) Ty.p} _. R? is the rotation about the x-axis through an angle and TpR? _> R? i s the rotation about the 

z-axis through an angle 9 2 . 


23. 

(a) Ty.R} _. p} is the reflection about the x-axis, and Tr.R? _ ► R? i s the reflection about the y-axis. 

(b) t’j p} _ , p} is the orthogonal projection on the x-axis, and Tr.R? _ ► R? i s the counterclockwise rotation 

through an angle 9 . 

(e) . p- is a dilation by a factor k, and Tp.R? _* R? is the counterclockwise rotation about the z-axis 

through an angle 9 . 


Discussion 

Discovery 


24. 


Indicate whether each statement is always true or sometimes false. Justify your answer by 
giving a logical argument or a counterexample. 


(a) If T.R? _> R? is the orthogonal projection onto the x-axis, then T~^ .R? _> R? 

maps each point on the x-axis onto a line that is perpendicular to the x-axis. 

(b) If T \: U _» V and TrV _+ W are linear transformations, and if 7 ^ is not 

one-to-one, then neither is Ri a T\- 






















(c) In the xy-plane, a rotation about the origin followed by a reflection about a coordinate 
axis is one-to-one. 


Does the formula T(a, b, c ) = ax 2 4- bx + c define a one-to-one linear transformation from p}' 

25 

to p-Jl Explain your reasoning. 

Let E be a fixed 2x2 elementary matrix. Does the formula 7(A) — EA define a one-to-one 
26* linear operator on Mji*- Explain your reasoning. 

Let a be a fixed vector in p}. Does the formula 7Y V ) = a x v define a one-to-one linear 
27* operator on p}7 Explain your reasoning. 

28. (For Readers Who Have Studied Calculus) The Fundamental Theorem of Calculus implies 
that integration and differentiation reverse the actions of each other in some sense. Define a 
transformation [)p n —* p n _ { by D(P(x)) = p ! (x), and define J\P n _ { —* P n by 

I p(t)dt- 
JO 


(a) Show that D and J are linear transformations. 


(b) Explain why J is not the inverse transformation of D. 

(c) Can we restrict the domains and/or codomains of D and J such that they are inverse 
linear transformations of each other? 
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8.4 

MATRICES OF GENERAL 
LINEAR 

TRANSFORMATIONS 


In this section we shall show that if V and W are finite-dimensional vector 
spaces (not necessarily R n and R m ), then with a little ingenuity any linear 
transformation T:V-*W can be regarded as a matrix transformation. The 
basic idea is to work with coordinate vectors rather than with the vectors 
themselves. 


Matrices of Linear Transformations 


Suppose that V is an n-dimensional vector space and W an m-dimensional vector space. If we choose bases B and B 1 for V and 
W, respectively, then for each x in V, the coordinate vector [ x ] B will be a vector in R n . and the coordinate vector [T(x) ] %< will 
be a vector in R m (Figure 8.4.1). 


- \ 

A vector 

in V 

(M-dimensional) 


A vector 
in R" 



x 


/ 




T(X) 




A vector 


1 


in W 

(m-dimcnsional) 


A vector 
in R m 


Figure 8.4.1 

Suppose T\ V —> W >s a linear transformation. If, as illustrated in Figure 8.4.2, we complete the rectangle suggested by Figure 
8.4.1, we obtain a mapping from R H to R m , which can be shown to be a linear transformation. (This is the correspondence 
discussed in Section and 4.3 we studied linear transformations from .) If we let A be the standard matrix for this transformation, 
then 




(1) 


The matrix A in 1 is called the matrix for T with respect to the bases B and B 1 . 


T maps 
V into W 

x ^ rtxi 




A 

s\ 


■ irnoL 


MiilliplicjiUon 
by A 

mops R H into R'" 


Figure 8.4.2 

Later in this section, we shall give some of the uses of the matrix A in 1, but first, let us show how it can be computed. For this 
purpose, let B = (uj, u 2 ,u H ) be a basis for the n-dimensional space V and B r = (vi, v 2 ,v w ) a basis for the 
m-dimensional space W. We are looking for an m x ^ matrix 
































A = 


a 11 <*12 - a ln 

a 21 <*22 - « 2 h 

a m 1 fl m2 L,J fl mn 


such that 1 holds for all vectors x in V, meaning that A times the coordinate vector of x equals the coordinate vector of the image 
T(x) of x. In particular, we want this equation to hold for the basis vectors uj, 112 ,..., u H ; that is, 


A\u\ 1 R = fTfuOl R ' 


A\ii'A r= \T(u 2 ) 1 r' 


A\u»]n = |T(ll„)l R' 


But 


so 



V 


’o’ 


’0’ 


0 


1 


0 

[ni]* = 

0 

[112] b = 

0 

[««]£ = 

0 


0 


0 


1 


^[«l] B = 


^[112] B = 


a n a 12 
«21 «22 

a nA a m2 

a n a 12 
-321 a 22 

a m\ a m2 


a 1h 

fl 2H 


df i H 
a 2 H 


All 

a 2l 

a ml 

a\2 

^22 

a m2 


( 2 ) 


Substituting these results into 2 yields 

'a n 


<*21 
a m\ 




an fli2 

<*21 <*22 

a m\ a m2 


= [7(ui)] F ', 


<*12 
<*22 

a m2 


a 1h 
a 2n 


a 


mn 


dfi H 
a 2 H 


a 


= [T(xx 2 )]b’ . 


mn 


a ln 

a 2n 

fijyxyx 


= [T(xx»)] b < 


which shows that the successive columns of A are the coordinate vectors of 

T(xx 1 ), T(u 2 ) . T{xx n ) 

with respect to the basis B 1 . Thus the matrix for T with respect to the bases B and B' is 

^=[[7’(ui)] F 'I[T(u2)]^H[T(u h )] F '] 


(3) 


This matrix will be denoted by the symbol 


so the preceding formula can also be written as 


\T\b\b 


[T] b',B = [ [ 7 ( 111 ) ] B <\ [7(112) ] B ’\-‘\ [ 7 (u„) ] s'] 


(4) 


and from 1, this matrix has the property 




































(4a) 


Observe that in the notation [T] gig the right subscript is a basis for the domain of T, and the left subscript is a basis 
for the image space of T (Figure 8.4.3). Moreover, observe howthe subscript B seems to “cancel out” in Formula 4a (Figure 
8.4.4). 
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L J 


Basis for ihe 


Basis for ihc 

| linage space 


domain 


Figure 8.4.3 
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t ;sn l!C 11.1 ’ 3>. III 


Figure 8.4.4 


Matrices of Linear Operators 

In the special case where y = W (so that T\ V —> y is a linear operator), it is usual to take 3 = B ! when constructing a matrix 
for T. In this case the resulting matrix is called the matrix for T with respect to the basis B and is usually denoted by [T] g 
rather than [ T] g g. If B = {uj, 112 ,..u H }, then Formulas 4 and 4a become 

[T] s =[ [TXui)] B \[T(u 2 )] *1-1 [T(u„)] *] (f 

and 

[T] B [x] B =[T(x)] B (5 , 

Phrased informally, 4a and 5a state that the matrix for T times the coordinate vector for x is the coordinate vector for T(x)- 


Matrix for a Linear Transformation 


Let T.P\ _ * P 2 linear transformation defined by 

T{p{x))=xp{x) 


Find the matrix for T with respect to the standard bases 

B= {ui,u 2 } 

where 


ui = l, u 2 = *; 


and B } = {vi, v 2 , V 3 } 
vi = 1 , \’2 = x, \'3 = x 2 


Solution 


From the given formula for T we obtain 













r(ui) = r(i) = co(i) =* 

T(u 2 ) = T(x) = (x) (x) = x 2 

By inspection, we can determine the coordinate vectors for T(ui) and T{ 112 ) relative to B’\ they are 



0 


0 


1 

[nu 2 )] B '= 

0 


0 


1 

Thus the matrix for T with respect to B and B f is 



'0 

[7’] B ' iB =[[r(ni)] B d[7’(n 2 )] B '] = 

1 




0 


♦ 


Verifying Formula (4a) 


Let t-.Pi 


P 2 be the linear transformation in Example 1. Show that the matrix 


[T\b'.b = 


0 0 
1 0 
0 1 


(obtained in Example 1) satisfies 4a for every vector x = a 4 - bx in Py 


Solution 


Since x = p(x) = a 4 Ax. we have 


T(x) =xp(x) =ax 4 bx 2 


For the bases B and B 1 in Example 1, it follows by inspection that 






'o' 

Wb = Wx-\ b] B = 

" a " 
b_ 

and 

[T(x)]^=[ fl x4 bx 2 ] = 

a 

b 


Thus 


B = 

1 - 

O O 

O 1 — 1 

1_ 

"lJ" 

b 


0 1 



0 

a 

b 


[T(x)] b > 


so 4a holds. 


♦ 


Matrix for a Linear Transformation 


Let T:R 2 _> P? be the linear transformation defined by 


*1 

x 2 


x 2 

— 5xi 4 13*2 

— 7xi4 16x2 


Find the matrix for the transformation T with respect to the bases B = {ui, 112} f° r R 2 and B > = {vi, \'2, V 3 } for pp, where 






























Ill = 






1 


- 1 


0 

3 

1 

= 

'5' 

2 

vi - 

0 

v 2 = 

2 

v 3 — 

1 





- 1 


2 


2 


Solution 

From the formula for 7, 



1 


2 

7(ui) = 

1 

'-n 

1 1 

_ 1 

. 7(112) = 

1 

-3 


Expressing these vectors as linear combinations of vj, \’ 2 , and V 3 , we obtain (verify) 

7Xui )= vi - 2 v 3, 7(112) = 3vi F v r 2 - V3 

Thus 


so 



1 



3 

[7(111)]*' = 

0 

[7(112)]*' 

■ — 

1 


-2 



- 1 




1 

3 

[7]*' f *=[[7(ui)]* 

'1 [7(112)]*'] = 

0 

1 


2 - 1 


EXAMPLE 4 Verifying Formula (5a) 


Let T\B? _> P? be the linear operator defined by 


and let B = { 111 , 112 } be the basis, where 


(pii 


xi+x 2 

b\ 

J- 

— 2x\ -1-4x2 


ill = 


™2 = 


(a) Find [7] %■ 

(b) Verify that 5a holds for every vector x in p}. 


Solution (a) 

From the given formula for 7, 


7(ui)= 


= 2ui, 7(ii2) = 


= 3ii2 


Therefore, 





































[T( «!)]* = 


and [rCu2)]B = 


Consequently, 


[7 , ] b =[[T(ui)] b 


[TM] B ] = 


2 0 
0 3 


Solution (b) 

if 


X = 


*1 

*2 


is any vector in $2, then from the given formula for T, 

T(x) = 


*l + *2 
— 2xi 4 4x2 


To find [ x] % and [T(x) ] g we must express 6 and 7 as linear combinations of uj and 113 . This yields the vector equations 


*1 

x 2 


= k 1 


I- ^2 


xi lx 2 
— 2xi 4 4x2 


= ci 


T c 2 


Equating corresponding entries yields the linear systems 

*1 4- k 2 =x\ 
*1 4 2k2 = X2 


and 


Solving 10 for ^ and yields 


so 


and solving 11 for cj and c 2 yields 


so 


ci 4- c 2 = xi 4- x 2 
ci 4- 2c2 = — 2xi 4- 4x2 


( 6 ) 

(7) 

( 8 ) 

(9) 

( 10 ) 

(ID 


2 xi -*2 
-xi |-x 2 


k\ = 2x\ — X 2 , jt 2 =-xi 4 -X 2 

ci =4 xi- 2x2, C2=-3 xi4 3x2 

m±)] B = 


4xi - 2 x 2 
- 3xi 4 3 x 2 


Thus 


so 5a holds. 


[T] B [x] B = 


to 

0 

2xi — x 2 


4xi - 2x2 

_0 3_ 

— xi 4 -x 2 


- 3xi + 3x2 


= [7Xx)]* 


Matrices of Identity Operators 

The matrix for the identity operator on V always takes a special form. 



































Matrices of Identity Operators 


If B= {ui, u 2> u M } ' s a basis for a finite-dimensional vector space V and y —> y is the identity operator on V, then 

/(ui)=ui, /(n 2 ) = 112 , , /(u H )=u H 

Therefore, 



V 


V 


V 


0 


l 


0 

[/(Ul)]* = 

0 

[IM] b = 

0 

[/("«)]* = 

0 


0 


0 


l 


Thus 


U] B = 


10-0 
0 1-0 

0 0-0 

0 0-1 


=1 


Consequently, the matrix of the identity operator with respect to any basis is the n x n identity matrix. This result could have 
been anticipated from Formula 5 a, since the formula yields 

m B [x] B =[i(x)] B =[x] B 

which is consistent with the fact that [/] s = /. 


We leave it as an exercise to prove the following result. 


THEOREM 8.4.1 


IfT.Rf —* R m is a linear transformation, and ifB and B* are the standard bases for R n and R m , respectively, then 

[T] b \b=[T] 


This theorem tells us that in the special case where T maps R n into Rf\ the matrix for T with respect to the standard bases is the 
standard matrix for T. In this special case, Formula 4a of this section reduces to 

[T] x = T(x) 


Why Matrices of Linear Transformations Are Important 

There are two primary reasons for studying matrices for general linear transformations, one theoretical and the other quite 
practical: 


Answers to theoretical questions about the structure of general linear transformations on finite-dimensional vector spaces 
can often be obtained by studying just the matrix transformations. Such matters are considered in detail in more advanced 
linear algebra courses, but we will touch on them in later sections. 


These matrices make it possible to compute images of vectors using matrix multiplication. Such computations can be 
performed rapidly on computers. 















To focus on the latter idea, let T\ V —* W be a linear transformation. As shown in Figure 8.4.5, the matrix [T] B > B can be used 
to calculate T(x) in three steps by the following indirect procedure: 


Compute the coordinate vector [ x ] ^. 


Direct 

X ——— - 7(xj 

eonipul niton 


(I) 


Multiply hv rn fl - H 

Figure 8.4.5 


(3) 




2. Multiply [ x ] s on the left by [T] B ^ B to produce [T(x) ] s >. 


Reconstruct T{\) from its coordinate vector [T(x) ] B i. 


Linear Operator on p 2 

Let T.Pj _ * Pi li near operator defined by 

T{p{x))=p{ 3x-5) 

that is, T(cq + c\x ■+ C 2 X 2 ) =co + ci(3x — 5) 4- C2(3x — 5) 2 - 
Find [7] s with respect to the basis B= |l, x, x 2 |. 

(b) Use the indirect procedure to compute T{\ -\-2x 3x 2 ). 

(c) Check the result in (b) by computing 7(1 4 2x 4- 3x 2 ') directly. 


Solution (a) 

From the formula for T, 


7(1) = 1, 700=31-5, T{x 2 ) = { 3x-5) 2 =9x 2 -30x + 25 


[T(V)] b = 

t 

0 

[T(x)]b = 

1 

1 

oo ui 
_1 

[ T(x 2 )] b = 

25' 

-30 


0 


0 


9 


Thus 
















Solution (b) 

The coordinate vector relative to B for the vector p = 1 -|- 2x 4- 3x 2 is 


Thus, from 5a, 


from which it follows that 


[P]fl = 


1 

2 

3 


[T(1H 2x + 3x 2 )] b =[T( V )] b =[T] b [ V ] b 


'1 

-5 

25' 

V 


66' 

0 

3 

-30 

2 

= 

-84 

0 

0 

9 

3 


27 


7(1 + 2x + 3x 2 ) = 66- 8% + 27x 2 


Solution (c) 

By direct computation, 

7(1 + 2* + 3x 2 ) = 1 + 2(3* - 5) + 3(3* - 5) 2 

= 1 + 6x - 10 + 27x 2 - 90* + 75 
= 66- 84* + 27x 2 

which agrees with the result in (b). 


# 


Matrices of Compositions and Inverse Transformations 

We shall now mention two theorems that are generalizations of Formula 21 of Section 4.2 and Formula 1 of Section 4.3. The 
proofs are omitted. 


THEOREM 8.4.2 


If T\:U 

_* y and J 2 ' V 

if are linear transformations, and ifB, B u , and B 1 are bases for U, V, and W, respectively, 

then 


[T 2 aT\] B *' B = [T2\b\B"[T\]b",B 

(13) 

THEOREM 8.4.3 

IfT.V 

~>Y is a linear operator, and ifB is a basis for V, then the following are equivalent. 


















(a) T is one-to-one. 


(b) [ T] £ is invertible. 


Moreover, when these equivalent conditions hold, 


['r%=[r],-' 

(14) 


Remark In 13, observe how the interior subscript B " (the basis for the intermediate space V) seems to “cancel out,” leaving 
only the bases for the domain and image space of the composition as subscripts (Figure 8.4.6). This cancellation of interior 
subscripts suggests the following extension of Formula 13 to compositions of three linear transformations (Figure 8.4.7). 

[7\ * r, ] fl . * = [7\] fl K IT1# • k 

t t 

h Lt m n 


Figure 8.4.6 

r, a T) 



Ikisis it iJasisi //" Basi*/?"' 

Figure 8.4.7 

[T3 o T2 o T\]b ! ,B = B' ,B !II [T 2 ] B !!! b" ,b 

The following example illustrates Theorem 8.4.2. 


Using Theorem 8.4.2 


Let T\\P\ _> P'i b e the linear transformation defined by 

T’iCT’OO) =xp (;0 

and let T 2 :P 2 _ * Pi the '> near operator defined by 

T 2 (p(x))=p(.3x-5) 

Then the composition (T 2 o T\) :P\ —* P 2 is given by 

<Jl o T x ){p{x)) = T 2 {T\(p(x))) = T 2 (xp(x)) = {2x - 5)p{2x - 5) 

Thus, if p(x) =co + c\x, then 

(T 2 o Ti)(c 0 -\-cix) = (3x-5)(c 0 + ci(3x-5)) 

= cq(3x — 5) 4- c\(3x — 5) 2 (K 

In this example, p j plays the role of U in Theorem 8.4.2, and pj plays the roles of both V and IT; thus we can take B f = B !! in 
13 so that the formula simplifies to 











[?2 ° T\ ]b',B = [T2]b , ['?’i]b i ,B 


( 17 ) 


Let us choose B = {1, x) to be the basis for and choose B l = |l, x, x 2 | to be the basis for We showed in Examples 
Example 1 and Example 6 that 


[T\ ]b\B = 

'0 O' 
1 0 

and [T 2 ] B ' = 

'1 - 5 25' 

0 3-30 


0 1 


0 0 9 


Thus it follows from 17 that 


[?2 o T\] B ^ B = 


'1 

-5 

25' 

'0 

O' 


' -5 

25' 

0 

3 

-30 

1 

0 

= 

3 

-30 

0 

0 

9 

0 

1 


0 

9 


(18) 


As a check, we will calculate [T 2 o T\] B : , B directly from Formula 4. Since B = {1, x }, it follows from Formula 4 with Ul = ] 
and U 2 = x that 


[Tl o T X ] B ' B = [ [ (T 2 a TO (1) ] B '\[ (T 2 o TO (x) ] B >] 


(19) 


Using 16 yields 

(T 2 oT0(1) = 3x-5 and (T 2 o TO (x) = (3x - 5) 2 = 9x 2 - 30x 4- 25 
Since B r = j 1, x, x 2 1 it follows from this that 


Substituting in 19 yields 


[(T 2 oTO(l)] B ' = 

-5 

3 

and [(T 2 oTO(x)] B ' = 

25 

-30 


0 

'-5 25' 

9 


[T 2 o T\] b - b = 


■30 

9 


which agrees with 18. 


Exercise Set 8.4 


Click here for Just Ask! 




















1 . 


Let T.Pi _» P 3 be the linear transformation defined by T(p(x)) = xp(x). 


(a) Find the matrix for T with respect to the standard bases 


B = |ui, 112 , 113 1 and B* = |vi, V 2 , Y3, V 4 I, 

where 

111 = 1 , 112=*, 113 =x 2 

VI = 1 , V2 = x, V3=x 2 , \'4 = x 3 


(b) Verify that the matrix [T] g-g obtained in part (a) satisfies Formula 4a for every vector x = CQ 4 \ c ^ x 2 in p 2 


Let T.P 2 —* P\ be the linear transformation defined by 

2 

T{a$ I ^1* I ) = Oo F^i) — (2a\ 4 3^2)^ 

^ a ) Find the matrix for T with respect to the standard bases B = jl, x, x 2 | and B = 11, x j for P 2 and Py 


(b) Verify that the matrix [T] g-g obtained in part (a) satisfies Formula 4a for every vector x = eg I- c\x + C 2 X 2 ' n P '1 


Let T. Pi —* P 2 be the linear operator defined by 

T{an 4 a\x 4 ajx 2 ) — an Fai O - 1) 4 a^(x — l ) 2 

Find the matrix for T with respect to the standard basis B = jl, x, x 2 | for py 


(b) Verify that the matrix [T] g obtained in part (a) satisfies Formula 5a for every vector * = \ a\x \ aix? in Pj- 


Let t p 2 

4. 


p 2 be the linear operator defined by 


T 


fF‘1 

1 _ 

1 

1 

* 

to 

_1 

W 

r 

1 

to 

1 _ 


and let B = {uj, 112 } be the basis for which 


111 = 


1 

1 


and 


112 = 


- 1 
0 


(a) Find [7] B - 

(b) Verify that Formula 5a holds for every vector x in p 2 . 


Let pp 2 _j. p 2 be defined by 


5 . 



pi 


XI A 2X2 

_ 

-*1 

IL^ 2 . 




0 


(a) Find the matrix [ T ] B > B with respect to the bases B = {uj, 112 } and B* = | v l> v ?, v 3 }, where 


ui = 

O' 

3 

«2 = 

'- 2 ' 

4 

vi = 

O' 

1 

▼2 = 

' 2 ' 

2 

v 3 = 

O' 

0 






1 


0 


0 


(b) Verify that Formula 4a holds for every vector 

'*f 

|/ 2 _ 

in r 2 . 

Let T.R? _> B? be the linear operator defined by 

T{x\, X 2 , * 3 ) = Ol -X2,^2-x\,x\ —xi) ■ 

(a) Find the matrix for T with respect to the basis B = (vj, V 2 , V 3 }, where 

VI = (1,0,1), V2 = (0,1,1), V3 = (1,T,0) 

(b) Verify that Formula 5a holds for every vector x:=(xi,x 2 ,X 3 )in j fi 3 . 

(c) Is T one-to-one? If so, find the matrix of 


7. 


Let T.P 2 _* P 2 b e the linear operator defined by T(p{x))=p(2x | 1)—that is, 

T(cq I C\X C2X 2 ) =CQ I Cl(2x-|-1) \-C2(2x hi) 2 


Find [ T] B with respect to the basis B = |l, x, x 2 j. 


(b) Use the indirect procedure illustrated in Figure 8.4.5 to compute 7(2 — 3x 4 - 4x 2 ). 


(c) Check the result obtained in part (b) by computing T {2 — 3x | Ax 2 ) directly. 


Let 7 ; P 2 _* P 2 be the linear transformation defined by T(p(?;)) = xp(x — 3 )—that is, 

T(c 0 f :ix + c 2 x 2 ) = x(c 0 I ciO - 3) 4 c 2 (x - 3) 2 ) 

( a ) Find [ T] with respect to the bases B= |l, x, x 2 j and B f = |l, x, x 2 , x 3 j 


(b) Use the indirect procedure illustrated in Figure 8.4.5 to compute 7(1 f x — x 2 ). 


(c) Check the result obtained in part (b) by computing 7(1 | x — x 2 ) directly. 


9. 


Let vj = 


1 

3 


and \2 = 



and let 


A = 


1 3 
-2 5 


be the matrix for T.B? _* B? with respect to the basis B = {vj, v 2 } • 

(a) Find [7(vi)]*and [7(v 2 )] F 

(b) Find 7(vi) and 7(v 2 )- 


(c) 


Find a formula for 7 


*1 

*2 


(d) 


Use the formula obtained in (c) to compute 7 


10 . 


Let A = 
7 ' = {wi, 


3-210 
1 6 2 1 

-3 0 7 1 

w 2 , w 3 1 , where 


be the matrix of 7 ; j ? 4 


p} with respect to the bases B = (vj, v 2 , v 3 , V 4 } and 



'o' 


2' 


l' 


'6' 


1 


1 


4 


9 

VI = 

1 

v 2 = 

- 1 

v 3 = 

- 1 

v 4 = 

4 


1 


- 1 


2 


2 



'O' 


' -7' 


'-6' 

w'l = 

8 

w 2 = 

8 

w 3 = 

9 


8 


1 


1 


(a) Find [7( Vl )] F , [7(v 2 )] F s [ 7 (v 3 )] F s and [7(v 4 )] F - 


(b) Find 7(vi), 7(v 2 ), 7(v 3 ), and 7(v 4 ). 


T*f 

1 

*2 


*3 


1 

j 


Find a formula for 7 



Idl 


/rsi\ 


Use the formula obtained in (c) to compute 



11. Let A = 


3 - 1 
0 5 
2 4 


be the matrix of T.P 2 _> p 2 with respect to the basis B = {vj, v 2 , V 3 }, where VJ = 2x \ 3x 2 > 


v 2 = — 1 4- 3x 4- 2x 2 > V 3 = 3 I lx 4 - 2x 2 - 


(a) Find [T(vy)] B , [T(v 2 )] F and [T(v 3 )] F 


(b) Find 7 (vi),T(v 2 )* and 7 (v 3 ). 


(c) Find a formula for T(an + , x + a , !X 2 ). 

(d) Use the formula obtained in (c) to compute T(1 4 - x 2 ). 


Let T\\P\ _ * P 2 be the linear transformation defined by 

T’l (p(x))=xp(x) 

and let p 2: p 2 _> p 2 be the linear operator defined by 

T 2 (p(x))=p(2x hi) 

Let B = {1, x) and B* = \ 1, x, be the standard bases for p j and p 2 - 


(a) Find [T 2 <=> T\] B ^ B , [T 2 ] B <, and [T\] B \ B . 

(b) State a formula relating the matrices in part (a). 

(c) Verify that the matrices in part (a) satisfy the formula you stated in part (b). 


13 . 


Let T\ .P\ —. P 2 be the linear transformation defined by 

Ti(co 4- ci*) = 2co — 3c\x 

and let T 2 : P 2 —* P 2 be the linear transformation defined by 

T 2 (cq I c\xc 2 x^) = 3c$x3c\x^ 3c 2 x^ 

Let 5= {!,*}, 5" = |l,x,x 2 |,and B f = jl, x, x 2 , x 3 j. 


(a) Find [T 2 o T\] B < B , [T 2 \ B ^ B », and [T\] b",B- 




(b) State a formula relating the matrices in part (a). 


(c) Verify that the matrices in part (a) satisfy the formula you stated in part (b). 


Show that if T\ V —> W is the zero transformation, then the matrix of T with respect to any bases for V and W is a zero 

14. matrix. 

Show that if T: V —> V is a contraction or a dilation of V (Example 4 of Section 8.1), then the matrix of T with respect to 

15. any basis for V is a positive scalar multiple of the identity matrix. 

Let B = { vj, v r 2 , V 3 , V 4 } be a basis for a vector space V. Find the matrix with respect to B of the linear operator T; y — > y 
16* defined by T{\\ ) = V 2 > T(v 2 ) = V 3 . T(v 3 ) = T{\ 4 ) = vp 

Prove that if B and B* are the standard bases for R 71 and R m , respectively, then the matrix for a linear transformation 
17- T\R n —> R m with respect to the bases B and B ! is the standard matrix for T. 

18. (For Readers Who Have Studied Calculus) 

Let D.P'i _* P 2 b e the differentiation operator D(\\) = p'(x"). In parts (a) and (b), find the matrix of D with respect to the 

basis B = {pi, P 2 , P 3 } • 

(a) pi = l,p 2 =i, p 3 = x 2 

(b) Pl = 2. P 2 = 2 — 3x, P 3 = 2 — 3x V 8 x 2 

(c) Use the matrix in part (a) to compute D(6 — 6x 4 24x 2 ). 

(d) Repeat the directions for part (c) for the matrix in part (b). 

19. (For Readers Who Have Studied Calculus) 

In each part, B = {f 1 , f 2 , f 3 } is a basis for a subspace V of the vector space of real-valued functions defined on the real 
line. Find the matrix with respect to B of the differentiation operator £)■ y — > y. 


( a ) f 1 = 1> f2 = sinx. f 3 = cosx 

(b) = bf 2 = e x ,f 3 =e 2x 

(c) f 1=e 2x ,f 2 = X e 2x ,f 3 = x 2 e 2x 



(d) Use the matrix in part (c) to compute D(As ly ' -f b?:e~ Iy — I0x 2 e 2x ). 


Discussion 

Discovery 


Let V be a four-dimensional vector space with basis B, let W be a seven-dimensional vector space 
20. with basis B 1 , and let f. V —» W be a linear transformation. Identify the four vector spaces that 
contain the vectors at the corners of the accompanying diagram. 


Dined 


tn 


computation 


Tlx.) 

(3) 


T Multiply by I7'l jr n 

m* (2) ' v m] r 


Figure Ex-20 


In each part, fill in the missing part of the equation. 

21 . 


(a) 


[72 oTi]*•■*= [T 2 ]_'_ [Ti]g« g 


(b) 


? 

[T ’3 o 7’ 2 o T\] b ^ b = [T 3 ] _ [ ?2 ] B 7*1 ]#", b 


Give two reasons why matrices for general linear transformations are important. 

22 . 
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8.5 

SIMILARITY 


The matrix of a linear operator T:V->V depends on the basis selected for V. 
One of the fundamental problems of linear algebra is to choose a basis for V 
that makes the matrix for T as simple as possible—a diagonal or a triangular 
matrix, for example. In this section we shall study this problem. 


Simple Matrices for Linear Operators 


Standard bases do not necessarily produce the simplest matrices for linear operators. For example, consider the linear operator 
T-p} _, p^ defined by 


fpll 

v 

*1 + 

* 2 ' 

N 

r 

— 2xi + 

4x2 


and the standard basis B = (ej, f° r where 


«■! 


*■2 


By Theorem 8.4.1, the matrix for T with respect to this basis is the standard matrix for T; that is, 

[T]B=[T] = [neO\ne 2 )] 

From 1, 


T(ei) = 


1 

-2 


n* 2 ) = 


so 


[T\b = 

In comparison, we showed in Example 4 of Section 8.4 that if 

1 ' 


ui = 


1 


1 1 
2 4 


™2 = 


then the matrix for T with respect to the basis B ! — |ui, 112 1 is the diagonal matrix 

[T] B ' = 


2 0 
0 3 


( 1 ) 


( 2 ) 


(3) 


(4) 


This matrix is “simpler” than 2 in the sense that diagonal matrices enjoy special properties that more general matrices do not. 


One of the major themes in more advanced linear algebra courses is to determine the “simplest possible form” that can be 
obtained for the matrix of a linear operator by choosing the basis appropriately. Sometimes it is possible to obtain a diagonal 
matrix (as above, for example); other times one must settle for a triangular matrix or some other form. We will be able only to 
touch on this important topic in this text. 


The problem of finding a basis that produces the simplest possible matrix for a linear operator T\ V —» V can be attacked by 
first finding a matrix for T relative to any basis, say a standard basis, where applicable, and then changing the basis in a manner 
that simplifies the matrix. Before pursuing this idea, it will be helpful to review some concepts about changing bases. 

Recall from Formula 6 in Section 6.5 that if the sets B = {uj, 112,..., u„) and B' = |iij ,112,..., 11^ | are bases for a vector space 
V, then the transition matrix from B' to B is given by the formula 

(5) 

This matrix has the property that for every vector v in V, 


























^[v]b'= Mb 


(6) 


That is, multiplication by P maps the coordinate matrix for v relative to B f into the coordinate matrix for v relative to B [see 
Formula 5 in Section 6.5], We showed in Theorem 6.5.4 that P is invertible and p -1 is the transition matrix from B to B ! ■ 

The following theorem gives a useful alternative viewpoint about transition matrices; it shows that the transition matrix from a 
basis B f to a basis B can be regarded as the matrix of an identity operator. 


THEOREM 8.5.1 


IfB and B ! are bases for a finite-dimensional vector space V, and if p V -> y is the identity operator, then the transition 

matrix from B ! to B is U]b,B'- 


Suppose that B = {ui, 113 ,.... u H } and $ = ( u i, uj, | are bases for V. Using the fact that /( v ) = v for all v in V, it 

follows from Formula 4 of Section 8.4 with B and B 1 reversed that 

m B,B' = [W"!) 1 fl|[/(»2>] b\'\ [/«)] b] 

Thus, from 5, we have U]b,b : = p , which shows that U]b,b : is the transition matrix from B ! to B. 

■ 

The result in this theorem is illustrated in Figure 8.5.1. 


t 



Basis. ■= R 1 Basis = B 

Figure 8.5.1 

[/] b,b ] ' is the transition matrix from B ! to B. 


Effect of Changing Bases on Matrices of Linear Operators 

We are now ready to consider the main problem in this section. 


If B and B ! are two bases for a finite-dimensional vector space V, and if T: V —> V is a linear operator, what 
relationship, if any, exists between the matrices [T] B and [T] 


The answer to this question can be obtained by considering the composition of the three linear operators on V pictured in Figure 
8.5.2. 



V 


V 

Ba^is - U 


V 

Basis - If 


V 

Basis ~ B 


Figure 8.5.2 






In this figure, v is first mapped into itself by the identity operator, then v is mapped into T(y) by T , then T(v') is mapped into 
itself by the identity operator. All four vector spaces involved in the composition are the same (namely, V); however, the bases 
for the spaces vary. Since the starting vector is v and the final vector is T’(v), the composition is the same as T; that is, 

T=IoToI (7) 

If, as illustrated in Figure 8.5.2, the first and last vector spaces are assigned the basis B 1 and the middle two spaces are assigned 
the basis B, then it follows from 7 and Formula 15 of Section 8.4 (with an appropriate adjustment in the names of the bases) that 

U"\b',B ,= 7*c-H = U] B 1 ,b[T~\ B,bU] B,B* /o\ 


or, in simpler notation, 


[T] b <=[I] b >,b[T]bV]b,b-' 


(9) 


But it follows from Theorem 8.5.1 that [/] BB r is the transition matrix from ff to B and consequently, [/] B \ B is the transition 
matrix from B to B ! ■ Thus, if we let P = [/] 35 ', then P _1 = [/] B t B , so 9 can be written as 

[T] B '=P~ l [T] B P 

In summary, we have the following theorem. 


THEOREM 8.5.2 


Let T\ V —> V be a linear operator on a finite-dimensional vector space V, and let B and B 1 be bases for V. Then 

[T] B :=P~ l [T] B P (10) 

where P is the transition matrix from B 1 to B. 


Warning When applying Theorem 8.5.2, it is easy to forget whether P is the transition matrix from B to B ! (incorrect) or from 
B ! to B (correct). As indicated in Figure 8.5.3, it may help to write 10 in form 9, keeping in mind that the three “interior” 
subscripts are the same and the two exterior subscripts are the same. Once you master the pattern shown in this figure, you need 
only remember that P = [/] B , B - is the transition matrix from B ! to B and that p -1 = [/] B < B is its inverse. 

[ f ljr= U Ift ft [ I ljj a- 

i A A 11 

The 

ifstenoc 

suhscnpEk 
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r he csterint 
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Figure 8.5.3 












Using Theorem 8.5.2 


T 


[pll 

v 

*1 + 

*2~ 

h\ 

r 

— 2x\ + 

4x2 


Let T\R 2 _» B? be defined by 


Find the matrix of T with respect to the standard basis B = {ei, * 2 ) f° r P?' then use Theorem 8.5.2 to find the matrix of T with 
respect to the basis B ! — jiq, j, where 

u 'l = 


V 

and U2 = 

V 

_1_ 


_2_ 


Solution 

We showed earlier in this section [see 2] that 


[T] b = 


1 1 
-2 4 


To find [ T] B - from 10, we will need to find the transition matrix 

= [/] = [ [**'l ] b| ["2 ] s] 

[see 5]. By inspection, 

u'l = ©l H- 
112 = ei 4- 2*2 


so 


[<1b = 


and [i4]b = 


Thus the transition matrix from B 1 to B is 


The reader can check that 


P = 


P _1 = 


1 1 
1 2 


2 - 1 
-1 1 


so by Theorem 8.5.2, the matrix of T relative to the basis B 1 is 


[T] B -' = P~ l [T] B P = 


2 - r 

1 r 

'1 r 


'2 

o' 

1 _ 

— 1 

^r 

■"j 

1 

_1 

.1 2 . 


_0 

3_ 


which agrees with 4. 


Similarity 

The relationship in Formula 10 is of such importance that there is some terminology associated with it. 


DEFINITION 


If A and B are square matrices, we say that B is similar to A if there is an invertible matrix P such that b = P~^AP- 



































It is left as an exercise to show that if a matrix B is similar to a matrix A, then necessarily A is similar to B. Therefore, 
we shall usually simply say that A and B are similar. 


Similarity Invariants 


Similar matrices often have properties in common; for example, if A and B are similar matrices, then A and B have the same 
determinant. To see that this is so, suppose that 

B = P~ { AP 


Then 


We make the following definition. 


det(5) = det(P _1 jlP) = det(F _1 )det( J 4)det(P) 

= ^^d*(A)det(P)=det(A) 


DEFINITION 


A property of square matrices is said to be a similarity invariant or invariant under similarity if that property is shared by 
any two similar matrices. 

In the terminology of this definition, the determinant of a square matrix is a similarity invariant. Table 1 lists some other 
important similarity invariants. The proofs of some of the results in Table 1 are given in the exercises. 

Table 1 

Similarity Invariants 


Property 

Description 

Determinant 

A and jpp have the same determinant. 

Invertibility 

A is invertible if and only if P~^AP is invertible. 

Rank 

A and j^p have the same rank. 

Nullity 

A and p~ l AP h ave the same nullity. 

Trace 

A and p ~ 1 jpp have the same trace. 

Characteristic 

polynomial 

A and P~^AP have the same characteristic polynomial. 

Eigenvalues 

A and p~ l AP h ave the same eigenvalues. 

Eigenspace 

dimension 

If h is an eigenvalue of A and p~ l AP^ then the eigenspace of A corresponding to A and the 
eigenspace of p~ l AP corresponding to A have the same dimension. 


It follows from Theorem 8.5.2 that two matrices representing the same linear operator p- V —► V with respect to different 




















bases are similar. Thus, if B is a basis for V, and the matrix [ T\ s has some property that is invariant under similarity, then for 
every basis B 1 , the matrix [ 7] R ; has that same property. For example, for any two bases B and B 1 we must have 

det([T] s )=det([T] s O 

It follows from this equation that the value of the determinant depends on T, but not on the particular basis that is used to obtain 
the matrix for T. Thus the determinant can be regarded as a property of the linear operator T; indeed, if V is a finite-dimensional 
vector space, then we can define the determinant of the linear operator T to be 

det(T) = det([7] B ) nn 


where B is any basis for V. 


Determinant of a Linear Operator 


Let T\B? _> B? be defined by 


fpll 

v 

*1 + 

*2" 

w 

r 

— 2xi + 

4x2 


Find det(7> 


Solution 


We can choose any basis B and calculate det( [T] %)• If we take the standard basis, then from Example 1, 


[T] b = 


1 1 
2 4 


so det(7) — 


1 1 
2 4 


= 6 


Had we chosen the basis B ! = |ui, 112 } of Example 1, then we would have obtained 


[T] B ' = 

which agrees with the preceding computation. 


2 0 
0 3 


so det(7) — 


2 0 
0 3 


= 6 


Reflection About a Line 


Let / be the line in the xjy-plane that passes through the origin and makes an angle 0 with the positive x-axis, where 0 < 9 < tt. As 
illustrated in Figure 8.5.4, let T.F? _> B? be the linear operator that maps each vector into its reflection about the line l. 



(a) Find the standard matrix for T. 






















(b) Find the reflection of the vector x = (1, 2) about the line / through the origin that makes an angle of Q = k / 6 with the 
positive x-axis. 


Solution (a) 

We could proceed as in Example (Example 6) of Section 4.3 and try to construct the standard matrix from the formula 

[T] B =[T] = [T(e l )\T( e2 )] 

where B = (ej, @ 2 } i s the standard basis for p 2 . However, it is easier to use a different strategy: Instead of finding [7] s 
directly, we shall first find the matrix [T] B <, where 

5' = {u',u'} 

is the basis consisting of a unit vector u ' along l and a unit vector u ' perpendicular to l (Figure 8.5.5). 



Figure 8.5.5 


Once we have found [ 71 p-s we shall perform a change of basis to find [7] The computations are as follows: 

7(uj)=Uj and 7 ( 112 )=— u 2 
so 




and T[(vl 2 )] b < = 


0 

- 1 


Thus 


[T] B ' = 


1 0 
0 - 1 


From the computations in Example 6 of Section 6.5, the transition matrix from B 1 to B is 

cos 9 — sin# 


P ~ [ [ U 1 ] b\ t u 2 ] b] — 


sin^ cos 


9 


It follows from Formula 10 that 


Thus, from 12, the standard matrix for T is 

[T]=P[T] B >P~ { 


[T] B = P[T] B -P~ l 


cos 9 

— sin 9 

'1 

o' 

cos 9 

sin# 

sin 0 

cos 9 

0 

-1 

— sin# 

cos 9 


cos 2 9— sin 2 9 2sin0cos# 

2 sin 9 cos 9 sin 2 9 — cos 2 9 

cos 2 9 sin 2# 
sin 29 — cos 29 


( 12 ) 


Solution (b) 























It follows from part (a) that the formula for T in matrix notation is 

x \ _ cos 20 sin 29 
? J sin 20 — cos 29 


Substituting Q = ^ j 5 in this formula yields 


1 £ 

2 2 

£ 1 

2 2 


so 


Thus 7X1,2) = f-i-t 


T 


1 £} 

2 2 

V 



£ 1 

2 _ 



2 2 



2 


Eigenvalues of a Linear Operator 


♦ 


Eigenvectors and eigenvalues can be defined for linear operators as well as matrices. A scalar X is called an eigenvalue of a 
linear operator 7 V —> V if there is a nonzero vector x in V such that Tx = Ax- The vector x is called an eigenvector of T 
corresponding to X. Equivalently, the eigenvectors of T corresponding to X are the nonzero vectors in the kernel of A/ — 7 
(Exercise 15). This kernel is called the eigenspace of T corresponding to X. 


Eigenvalues of a Linear Operator 


Let V = F(— 00, do) and consider the linear operator T on V that maps f (x) to f (x — 2~) • If j (?;) = sm(x'), then 
T’(f) = sin(x — 2~) = sin(x ) 5 so sin(x) is an eigenvector of T associated with the eigenvalue 1: 

7 , (sin(x)) = 1 - sin(x) 

Other eigenvectors of T associated with the eigenvalue 1 include sm(2x), cos(5?;), and the constant function 3. 


♦ 


It can be shown that if V is a finite-dimensional vector space, and B is any basis for V, then 
The eigenvalues of T are the same as the eigenvalues of [ 7 ] B - 

A vector* is an eigenvector of T corresponding to X if and only if its coordinate matrix [ x ] s is an eigenvector of [T] g 
corresponding to X. 

We omit the proofs. 


Eigenvalues and Bases for Eigenspaces 



























Find the eigenvalues and bases for the eigenspaces of the linear operator T\ P'i _» Pj defined by 

T(a 4 bx 4 cx 2 ) = — 2c 4 (a 4 2b 4 - c)x 4 (a 4 3c)x 2 


Solution 

The matrix for T with respect to the standard basis B = jl,x,x 2 j is 

[T] b = 


0 0-2 
1 2 1 

1 0 3 


(verify). The eigenvalues of T are A = 1 and A = 2 (Example 5 of Section 7.1). Also from that example, the eigenspace of [T] B 
corresponding to A = 2 has the basis (m, 113 } > where 



■- 1 ' 


'o' 

111 = 

0 

*2 = 

1 


1 


0 


and the eigenspace of [T] B corresponding to A = 1 has the basis ( 113 }, where 

— 2 " 

1 


™3 = 


1 


The matrices uj, 113 , and 113 are the coordinate matrices relative to B of 

Pl=-l+x 2 , 1 > 2 =*> 1)3 = - 2 fi + i 2 

Thus the eigenspace of T corresponding to A = 2 has the basis 

{1M.P2} = | - 1 + x 2 , x| 

and that corresponding to A = 1 has the basis 

{P3} = |-2 + x-fx 2 | 

As a check, the reader should use the given formula for T to verify that T(i)i) = 2pp T{ P 2 ) = 2p2> an< ^ ^(P 3 ) = 1 > 3 - 


Diagonal Matrix for a Linear Operator 


Let T\P? _> P? be the linear operator given by 


f 

-xr 

\ 


— 2 x 3 

T 

x 2 


= 

xi 4 2x2 + *3 

L 

X3 

/ 


xi 4 3x3 


Find a basis for g} relative to which the matrix for T is diagonal. 


Solution 

First we will find the standard matrix for T\ then we will look for a change of basis that diagonalizes the standard matrix. 
If B= (ei, e 2 , £ 3 } denotes the standard basis for g}, then 




















1 0 0 0 p - 2 

7’Cei) = r 0=1. r(e 2 ) = r 1=2, r(e 3 ) = r o = l 


o 1 


W 


so the standard matrix for T is 


[71 = 1 2 1 (13) 

1 0 3 

We now want to change from the standard basis B to a new basis B ! — |iij, u^, 113 | in order to obtain a diagonal matrix for T. If 
we let P be the transition matrix from the unknown basis B ! to the standard basis B, then by Theorem 8.5.2, the matrices [ T] 
and [T] B < will be related by 

[T] B :=P~ X [T]P ( 14 ) 


In Example 1 of Section 7.2, we found that the matrix in 13 is diagonalized by 

'-1 0 -2' 

P= 0 1 1 

1 0 1 

Since P represents the transition matrix from the basis B ! = ju'J, 113 , U 3 | to the standard basis B = 
Pare [«i] B ' [ u 2 ] B' and [WIF 80 

"-ll To] [-2' 

[ U 1 ] B ~ 0 , [ ll 2 ] B = 1 > [ u 3 ] B = 1 

1 0 1 


, the columns of 


= C - I)*! 4 (0)« , 2 + (1)^3 = 



u 2 — (0)e 1 I- (l)e 2 T (0)e 3 = 1 


« 3 = ( “ 2)ei + (l)e 2 4- (l)e 3 = 1 


are basis vectors that produce a diagonal matrix for [T] B \ 

As a check, let us compute [T] B - directly. From the given formula for T, we have 




= 1 = ll'. 


so that 


[?Xui )]*•■ = 


[7’(n 2 ) ] B 1 ~ 2 
0 


[T( ii£)]*--= 0 
1 


[T] b > = [[ T( «i )] b ][T(v! 2 )] b .}[ T{ 11 ') ] £•■] 


= n ■? 


This is consistent with 14 since 






P~ l [T}P = 

1 

0 

2 ' 

'0 

0 

- 2 ' 

'-1 

0 

- 2 ' 

1 

1 

1 

1 

2 

1 

0 

1 

1 


-1 

0 

- 1 

1 

0 

3 

1 

0 

1 


2 0 0 
0 2 0 
0 0 1 


We now see that the problem we studied in Section 7.2, that of diagonalizing a matrix A, may be viewed as the problem of 
finding a diagonal matrix D that is similar to A, or as the problem of finding a basis with respect to which the linear 
transformation defined by A is diagonal. 


Exercise Set 8.5 



Click here for Just Ask! 


In Exercises 1-7 find the matrix of T with respect to the basis B, and use Theorem 8.5.2 to compute the matrix of T with respect 
to the basis B* ■ 


T:R 2 _> R? is defined by 


(M 

i- 

XI - 2X2 

IH 

r 

-X2 


B= {ui, n 2 } and B> = { V 1 ’ v 2 }> where 



T 


'o' 


' 2 ' 


' -3' 

ill = 

0 

«2 = 

1 

vi = 

1 

v 2 = 

4 


2 . 


T:R 2 


r2 is defined by 


T 


(T* 1 ! 


xi f 7x2 

(L^J 

J" 

3xi —4x2 


B= {ui, « 2 ) and = { V T v 2}> where 



'2' 


4' 


"r 


' - r 

ui = 

2 

= 

-1 

vi = 

3 

v 2 — 

_ -1 


T\F? _> P? is the rotation about the origin through 45°; B and B ! are the bases in Exercise 1. 


T\R? _» R 3 is defined by 


f 

-*r 

\ 


xi 4 2 x 2 -X3 

t\ 



= 

~ x 2 

L 

*3 

/ 


xi 4- 7x3 


B is the standard basis for p} and P ! = |vi, \ r 2 , V 3 \, where 



V 


V 


V 

vi = 

0 

▼2 = 

1 

v 3 = 

1 


0 


0 


1 




















































5 . 


T\F? _> B? is the orthogonal projection on the xy-plane; B and B 1 are as in Exercise 4. 


T\F? _> B? is defined by T(x) = 5x; B and B f are the bases in Exercise 2. 

6 . 

T.P \—> P\ is defined by T{o,q 4 a\x) =aQ-\-a\(x f 1); 5= {pi, i> 2 ) an d B' = {<li, < 12 where pj = 6 | 3x> 

^' i *2 = 10 4 2x> qi = 2, f \2 = 3 4- 2x- 

Find det(T)- 

8 . 

(a) T:B 2 —»■ R 2 , where T{x\, X2) = (3xi — 4 ^ 2 , — *1 I 7 x 2 ) 

(b) T.B? —> where T{x\, * 2 , X 3 ) = (xj — * 2 , x 2 — * 3 , X 3 — xj) 

( c ) T.P2—* Py w b ere T{p{x)) = p(x — 1 ) 

Prove that the following are similarity invariants: 

9. 

(a) rank 

(b) nullity 

(c) invertibility 

Let T.P4 _ * Pj\ be the linear operator given by the formula T(p(?:)) =p{ 2x f 1 ). 

(a) Find a matrix for T with respect to some convenient basis; then use Theorem 8.2.2 to find the rank and nullity of T. 

(b) Use the result in part (a) to determine whether T is one-to-one. 


In each part, find a basis for f' 2 relative to which the matrix for T is diagonal. 


(a) 


/pi] 


' *l-*2 

IW 

J 

2xi 4 4x2 


(b) 


T 


fpil 


4xi -*2 

N 

J - 

- 3xi + x 2 


11 . 


In each part, find a basis for £>} relative to which the matrix for T is diagonal. 

12 . 


( a ) /| 
T 


*1 

*2 

*3 


- 2xi + *2 -*3 
X! - 2x 2 -x 3 
-xi — x 2 — 2x 3 


(b) 



r*r 

1 


' ~*2 + *3 _ 

T 

*2 


= 

-xi 4 x 3 


[*3 

J 


xi 4 x 2 


(c) /| 

T 


*1 

*2 

*3 


\L 


4xi + x 3 
2xi I 3x2 I 2 x 3 
xi 4 4x 3 


Let T. P 2 —» P 2 b e defined by 

'• 2 2 
T(fln -I- ai* 4- aix*) — (5an f 6ai 4- 2 ^ 2 ) — fai 4- 8 a?)x 4- (an — 2a2)x 

(a) Find the eigenvalues of T. 

(b) Find bases for the eigenspaces of T. 

Let T- M 22 _* M 22 be defined by 

2c a F c 
b — 2c d 

(a) Find the eigenvalues of T. 

(b) Find bases for the eigenspaces of T. 


XT. 


a b 
c d 


Let X be an eigenvalue of a linear operator T\ V —> V- Prove that the eigenvectors of T corresponding to X are the nonzero 
15. vectors in the kernel of A 1-7- 


16 . 

(a) Prove that if A and B are similar matrices, then A 1 and B 2 are also similar. More generally, prove that A k and B k are 
similar, where k is any positive integer. 


(b) If A 1 and p 2 are similar, must A and B be similar? 


17 . 


Let C and D be m x n matrices, and let B = {vj, V 2 ,..v H } be a basis for a vector space V. Show that if C [ x ] g = D[x] 




for all x in V, then C = £■■ 


Let / be a line in the xy-plane that passes through the origin and makes an angle 0 with the positive a- ax is. As illustrated in 

18. the accompanying figure, let gg} _, g} be the orthogonal projection of gl onto l. Use the method of Example 3 to show 

that 


Note See Example 6 of Section 4.3. 



cos 2 8 
sin^cos# 


sin#cos 0 
sin 2 8 



Discussion 

Discovery 


Indicate whether each statement is always true or sometimes false. Justify your answer by giving a 
19. logical argument or a counterexample. 

(a) A matrix cannot be similar to itself. 


(b) If A is similar to B, and B is similar to C, then A is similar to C. 


(c) If A and B are similar and B is singular, then A is singular. 

(d) If A and B are invertible and similar, then yj -1 and 5 -1 are similar. 


20 . 


Find two nonzero 2x2 matrices that are not similar, and explain why they are not. 


21 . 


Complete the proof by filling in the blanks with an appropriate justification. 


Hypothesis: A and B are similar matrices. 

Conclusion: A and B have the same characteristic polynomial (and hence the same 
eigenvalues). 


Proof: 


(1) det(A/ — B) =det (M-P^AP) 


( 2 ) 


det {XP~ l P - P~ X AP) 


= det(P - 1 (A/-A)P) 


(3) 

















(4) 


= det(f ?-1 )det(A/ — A)det(.P)_ 

(5) = det(f ?-1 )det(/ ? )det(A/ — A)_ 

(6) = det(A7 - A) _ 

If A and B are similar matrices, say g — p~ 1 then Exercise 21 shows that A and B have the 
22- same eigenvalues. Suppose that X is one of the common eigenvalues and x is a corresponding 
eigenvector for A. See if you can find an eigenvector of B corresponding to X, expressed in terms 
of X, x, and P. 

Since the standard basis for R n is so simple, why would one want to represent a linear operator on 

23. p n in another basis? 

Characterize the eigenspace of \ = 1 in Example 4. 

24. 

Prove that the trace is a similarity invariant. 

25. 
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8 £ Our previous work shows that every real vector space of dimension n can be 

■ D related to R n through coordinate vectors and that every linear transformation 

ISOMORPHISM from a real vector space of dimension n to one of dimension m can be related 

to R n and R m through transition matrices. In this section we shall further 
strengthen the connection between a real vector space of dimension n and R n . 


Onto Transformations 

Let V and Wbe real vector spaces. We say that the linear transformation T. V —> W is onto if the range of T is W —that is, if for 
every w in W, there is a v in V such that 

T(v) = w 

An onto transformation is also said to be surjective or to be a surjection. For a surjective mapping, then, the range and the 
codomain coincide. 


Onto Transformations 


Consider the projection p-py _, p} defined by p{x, y, z ) = (x, y)- This is an onto mapping, because if w = (x, y) is a point in 

p}, then y = (x, y, 0) is mapped to it. (Of course, so are infinitely many other points in j? 3 .) 

Consider the transformation Q.R —> R defined by P(x, y, z ) = (x, y, 0)- This is essentially the same as P except that we 
consider the result to be a vector in p} rather than a vector in p 2 . This mapping is not onto, because, for example, the point (1,1, 
1) in the codomain is not the image of any v in the domain. 

♦ 

If a transformation p- y —> W is both one-to-one (also called injective or an injection ) and onto, then it is a one-to-one mapping 

to its range W and so has an inverse 7 ’ -1 : yy _, y. A transformation that is one-to-one and onto is also said to be bijective or to 

be a bijection between V and W. In the exercises, you'll be asked to show that the inverse of a bijection is also a bijection. 

In Section 8.3 it was stated that if V and W are finite-dimensional vector spaces, then the dimension of the codomain W must be at 
least as large as the dimension of the domain V for there to exist a one-to-one linear transformation from V to W. That is, there 
can be an injective linear transformation from V to W only if dim(F) < dim(fT)- Similarly, there can be a surjective linear 
transformation from Vto W only if dim(F) > dim(fT)- Theorem 8.6.1 follows immediately. 

THEOREM 8.6.1 


Bijective Linear Transformations 

Let V and W be finite-dimensional vector spaces. If &m{V) ^ dim(JT). then there can be no bijective linear transformation 
from V to W. 


Isomorphisms 

Bijective linear transformations between vector spaces are sufficiently important that they have their own name. 















DEFINITION 


An isomorphism between V and W is a bijective linear transformation from V to W. 


Note that if T is an isomorphism between V and W, then 7 ’ -1 exists and is an isomorphism between W and V. For this reason, we 
say that V and W are isomorphic if there is an isomorphism from V to W. The term isomorphic means “same shape,” so 
isomorphic vector spaces have the same form or structure. 

Theorem 8.6.1 does not guarantee that if dim(V") = dim (IT), then there is an isomorphism from V to W. However, every real 
vector space V of dimension n admits at least one bijective linear transformation to R n : the transformation T(v) = (v) $ that takes 
a vector in V to its coordinate vector in R n with respect to the standard basis for R n . 


THEOREM 8.6.2 


Isomorphism Theorem 

Let V be a finite-dimensional real vector space. If dim(F') = n> then there is an isomorphism from V to R n . 


We leave the proof of Theorem 8.6.2 as an exercise. 


EXAMPLE 2 An Isomorphism between p 3 and R A 


The vector space p 3 is isomorphic to r a , because the transformation 

T(a + xb 4 - ct?" + dx 3 ) = (a, b, c, d) 


is one-to-one, onto, and linear (verify). 


An Isomorphism between M 2 2 ar| d R 4 


The vector space M 22 i somor phic to pf, because the transformation 


T 


a ^ I = {a, b, c, d) 
c d J j 


is one-to-one, onto, and linear (verify). 


♦ 


The significance of the Isomorphism Theorem is this: It is a formal statement of the fact, represented in Figure 8.4.5 and repeated 
here as Figure 8.6.1 for the case y = fp, that any computation involving a linear operator Ton V is equivalent to a computation 
involving a linear operator on R M ; that is, any computation involving a linear operator on V is equivalent to matrix multiplication. 
Operations on V are effectively the same as those on R n . 














uitccl 


st 

g- 

■5 

3 




mmfuUalHSh 


Multiply by A 


Tt JO 


I / 1 x 1 Ijf 


Figure 8.6.1 


If dun(F) = m, then we say that V and R n have the same algebraic structure. This means that although the names conventionally 
given to the vectors and corresponding operations in V may differ from the corresponding traditional names in R n , as vector 
spaces they really are the same. 

Isomorphisms between Vector Spaces 

It is easy to show that compositions of bijective linear transformations are themselves bijective linear transformations. (See the 
exercises.) This leads to the following theorem. 


THEOREM 8.6.3 


Isomorphism of Finite-Dimensional Vector Spaces 

Let V and W be finite-dimensional vector spaces. If dim(F) = dim(PF)’ t ^ en V an< ^ W are isomorphic. 


Proof We must show that there is an isomorphism from V to W. Let n be the common dimension of V and W. Then there is an 

isomorphism T: V -> R n by Theorem 8.6.2. Similarly, there is an isomorphism S: W -> R n - Let R = £' _1 . Then R 0 7 is an 

isomorphism from V to W, so V and W are isomorphic. 


An Isomorphism between p 3 and M 22 


Because = 4 and dim(M 22 ) = 4’ these spaces are isomorphic. We can find an isomorphism T between them by 

identifying the natural bases for these spaces under T.P-i _> Mjf- 

1 O' 


7(1) = 
T(x) = 
T(x 2 )= 
T(x 3 )= 


0 0 

0 1 
0 0 

0 0 
1 0 

0 0 
0 1 


If p(x) = a V xb +- cx 2 + dx 3 is in p 3 , then by linearity, 




















T(j>(x))=a 


1 0 
0 0 


+ b 


0 1 
0 0 


4- c 


0 0 
1 0 


4 d 


0 0 
0 1 


a b 
c d 


This is one-to-one and onto linear transformation (verify), so it is an isomorphism between p 3 and M 22 - 


♦ 


In the sense of isomorphism, then, there is only one real vector space of dimension n, with many different names. We take R n as 
the canonical example of a real vector space of dimension n because of the importance of coordinate vectors. Coordinate vectors 
are vectors in R n because they are the vectors of the coefficients in linear combinations 

flivi 4- 

and since our scalars a are real, the coefficients (a\,a n ) are real n-tuples. 

Think for a moment about the practical import of this result. If you want to program a computer to perform linear operations, 
such as the basic operations of the calculus on polynomials, you can do it using matrix multiplication. If you want to do video 
game graphics requiring rotations and reflections, you can do it using matrix multiplication. (Indeed, the special architectures of 
high-end video game consoles are designed to optimize the speed of matrix-matrix and matrix-vector calculations for computing 
new positions of objects and for lighting and rendering them. Supercomputer clusters have been created from these devices!) This 
is why every high-level computer programming language has facilities for arrays (vectors and matrices). Isomorphism ensures 
that any linear operation on vector spaces can be done using just those capabilities, and most operations of interest either will be 
linear or may be approximated by a linear operator. 


Exercise Set 8.6 



Click here for Just Ask! 


Which of the transformations in Exercise 1 of Section 8.3 are onto? 

1 . 


Let A be an n x n matrix. When is T^.R n —> R> n not onto? 


Which of the transformations in Exercise 3 of Section 8.3 are onto? 

3. 


Which of the transformations in Exercise 4 of Section 8.3 are onto? 

4. 


Which of the following transformations are bijections? 

5. 


(a) T.P 2 {x) ^ P 2 {x),T{P{x))=xp{x) 

( b ) T.M 2 2 -* M 2 t T{A)=A T 

(c) T\R A - >R?, T(x,y,z,w) = (x,y, 0) 
















(d) T:Pj —* R 3 , T(a 4- bx 4- cx 2 ■+ dx 3 ) = (b, c, d) 


Show that the inverse of a bijective transformation from V to W is a bijective transformation from W to V. Also, show that the 

6 . inverse of a bijective linear transformation is a bijective linear transformation. 

Prove: There can be a surjective linear transformation from V to W only if dim(V) > dirn(ff')- 

7. 

8 . 

(a) Find an isomorphism between the vector space of all 3 x 3 symmetric matrices and pf J . 

(b) Find two different isomorphisms between the vector space of all 2 x 2 matrices and £; 4 . 

(c) Find an isomorphism between the vector space of all polynomials of degree at most 3 such that ^( 0 ) = 0 and r-\ 

(d) Find an isomorphism between the vector space span {1, sin(x), cos(x) } and R 3 . 

Let S be the standard basis for R n . Prove Theorem 8.6.2 by showing that the linear transformation T: V —* R }1 that maps vcV 
9. to its coordinate vector ( v ) s in R n is an isomorphism. 

Show that if T\, T 2 are bijective linear transformations, then the composition 7 2 o T\ is a bijective linear transformation. 


11. (For Readers Who Have Studied Calculus) 

How could differentiation of functions in the vector space span {1, sin(x), cos(^), sin(2;r), cos(2x)} be computed by matrix 
multiplication in R/l Use your method to find the derivative of 3 _ 4 sin(x) H sin(2x) I- 5 cos(2x)- 


Discussion 

Discovery 


Isomorphisms preserve the algebraic structure of vector spaces. The geometric structure depends 
on notions of angle and distance and so, ultimately, on the inner product. If V and W are 
finite-dimensional inner product spaces, then we say that T: V —> W is an inner product space 
isomorphism if it is an isomorphism between V and W, and furthermore, 

(u, v ) f =(7(u), T(v)) w 

That is, the inner product of u and v in V is equal to the inner product of their images in W. 


(a) Prove that an inner product space isomorphism preserves angles and distances—that is, the 
angle between u and v in V is equal to the angle between T’(u) and T(y) in W, and 

l|u-v|| r =||r(u)-7’(v)|| (r 







(b) Prove that such a T maps orthonormal sets in V to orthonormal sets in W. Is this true for an 
isomorphism in general? 

(c) Prove that if W is Euclidean n-space and if dirn(^) = n, then there is an inner product space 
isomorphism between V and W. 

(d) Use the result of part (c) to prove that if dim(fO = dim(fP)> then there is an inner product 
space isomorphism between V and W. 

(e) Find an inner product space isomorphism between and ^ 23 - 
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Chapter 8 


Supplementary Exercises 


Let A be an n x n matrix, B a nonzero n x 1 matrix, and x a vector in R n expressed in matrix notation. Is T(x") = Ax -f B a 
1* linear operator on R n l Justify your answer. 


Let 


2 . 


A = 


cos 9 
sin 0 


— sin# 
cos 9 


(a) Show that 


A 2 


cos 29 — sin 29 
sin 29 cos 29 


and A? 


cos 39 — sin 39 
sin 39 cos 39 


(b) Guess the form of the matrix a m for any positive integer n. 


(c) By considering the geometric effect of r-r 2 _» R 2 , where T is multiplication by A, obtain the result in (b) 

geometrically. 


Let vq be a fixed vector in an inner product space V, and let R: V —> V be defined by T(y) = (v, vqJvq- Show that Lis a 
*** linear operator on V. 

Let Vi, \ r 2 ,-■ v m be fixed vectors in R n , and let T\R n —» R m be the function defined by T(\) — (x - vi, x - ..., x - v m ) 

, where x ■ v,- is the Euclidean inner product on R n . 

(a) Show that T is a linear transformation. 

(b) Show that the matrix with row vectors vi, \’ 2 , • • • , \ m is the standard matrix for T. 

Let {e 1 ,e 2 ,e 3 ,e 4 } be the standard basis for j^ 4 , and let T:R 4 _» R 3 be the linear transformation for which 

5 ’ 7Xei) = (1,2,1), r(e 2 ) = (0,1,0), 

r(e 3 ) = (1-3,0), 7 ( 04 ) = (1,1,1), 


(a) Find bases for the range and kernel of T. 













(b) Find the rank and nullity of T. 


Suppose that vectors in p} are denoted by ] x 3 matrices, and define T.P? _* E? by 

2 4 
0 1 
2 5 


(a) Find a basis for the kernel of T. 

(b) Find a basis for the range of T. 

Let B = {vi, \' 2 , V 3 , V 4 } be a basis for a vector space V, and let T : y —* y be the linear operator for which 

7 . 

T(vi) = vi H \'2 4- V3 4- 3\'4 
T(y 2 ) = vi - V 2 f 2 v3 F 2v4 
T(\'2) — 2vj — 4\ r 2 4 5 v3 I 3\'4 
7X v 4) = — 2vi 4- 6 v 2 — 6\'3 — 2v4 

(a) Find the rank and nullity of T. 

(b) Determine whether T is one-to-one. 

Let V and W be vector spaces, let T, and be linear transformations from V to W, and let k be a scalar. Define new 
8 * transformations, T\ f T 2 an d kT, by the formulas 

(7l +T 2 )(x) = T l (x)+T 2 (x) 

(kT)(x)=k(T(x.)) 


(a) Show that (T’j 4 7 ’ 2 ): V -> W and kT\ V -> W are linear transformations. 



(b) Show that the set of all linear transformations from V to W with the operations in part (a) forms a vector space. 





Let A and B be similar matrices. Prove: 


9 . 


(a) A T and B T are similar. 


(b) If A and B are invertible, then A -1 and £? _1 are similar. 


10. (Fredholm Alternative Theorem) 

Let T: V -> V be a linear operator on an //-dimensional vector space. Prove that exactly one of the following 

statements holds: 


(i) The equation T(x) =h has a solution for all vectors b in V. 


(ii) Nullity of T > 0- 


11 . 


Let 7 : M 22 _* M 22 hnear operator defined by 


T(X) = 


1 1 
0 0 


X f X 


0 0 
1 1 


Find the rank and nullity of T. 


12 . 


Prove: If A and B are similar matrices, and if B and C are similar matrices, then A and C are similar matrices. 


13. 


14. 


Let L.M 22 _ * M 22 h e the linear operator defined by L(M) — Find the matrix for L with respect to the standard 

basis for ^ 22 - 

Let B — (ui, 112 , 113 } and = |v 1 , v r 2 , V 3 1 be bases for a vector space V, and let 


P = 


2 

1 

0 


-1 3 
1 4 
1 2 


be the transition matrix from B s to B. 

(a) Express \q, V2, V3 as linear combinations of 111,112,113. 

(b) Express up 112,113 as linear combinations of vp \'2, V3. 


15. 


Let B — {up U 2 , 113 } be a basis for a vector space V, and let T: V 


[T] b = 


-3 4 7 

10-2 
0 1 0 


Find [T] g>, where B ! = jvp V 2 , V 3 X is the basis for V defined by 


V be a linear operator such that 



VI =111, 


v 2 — m -f U2, 


v 3 = «1 + U2 -I- 113 


Show that the matrices 

2 f 

1 3 . 

are similar but that 

-1 2 
1 0 

are not. 



10. 


1 1 
1 4 


and 


Suppose that 7 y 

17. 


Find [T] B - 


y is a linear operator and B is a basis for V such that for any vector x in V, 



"^1 —*2 I-*!' 




[7(x)] fl = 

x 2 

*1 -*3 

if 

[i]i = 

x 2 

X'i 


Let T ■ V _ * y be a linear operator. Prove that T is one-to-one if and only if det(T) * 0- 

18. 


19. (For Readers Who Have Studied Calculus) 

(a) Show that iff = f (*), then the function D: C 2 ( — 00 , 00 ) — * F{ — 00 , 00 ) defined by D(f ) = / n (x) is a 
linear transformation. 

(b) Find a basis for the kernel of D. 

(c) Show that the functions satisfying the equation D(f) — / (x) fo rm a two-dimensional subspace of 
C 2 ( — 00 , 00 ), and find a basis for this subspace. 


Let r- p 2 

20 . 


p 3 be the function defined by the formula 


T{p{x)) = 


Pi- 1) 
7(0) 
7(1) 


(a) Find T(x 2 \ 5x + 6). 


(b) Show that T is a linear transformation. 

(c) Show that T is one-to-one. 



(d) Find 



(e) Sketch the graph of the polynomial in part (d). 


Let jq, x 2 , and x 3 be distinct real numbers such that *1 < X 2 < * 3 , and let p- p 2 
21 * formula 


T(p(x)) = 


P(x l) 
p(x 2 ) 
^(* 3 ) 


p? be the function defined by the 


(a) Show that T is a linear transformation. 

(b) Show that T is one-to-one. 


(c) Verify that if a\, a 2 , and a 3 are any real numbers, then 


where 


( 


\ 


a 2 


L 

a 3 

/ 


= a 1^1 (*) + 02^2 00 H fl 3^3(*) 


P lW = 


0 - xj) 0 ~ *3) 
C^i — ^2) C^i -*3) ’ 


^200 = 


0 —*i)Q —* 3 ) 

( x 2~ x \) ( x 2 — x 3) ’ 


P3(x) = 


(^ — ^1) (^ — ^2) 
C^3 — ^l) C^3 — ^2) 


(d) What relationship exists between the graph of the function 

a 1^1 OO + 32^200 4 <* 3^3 (*) 
and the points (jc 1>fll ), {x 2 ,a 2 ), and (* 3 , a 3 )? 






22. (For Readers Who Have Studied Calculus) 

Let p(x) and q(x) be continuous functions, and let Vbe the subspace of C( — oo , - 1 - oo ) consisting of all twice 
differentiable functions. Define py —*. y by 

^OOO) =/'(*) h p(x)y f (x) + <70M*) 


(a) Show that L is a linear transformation. 


(b) Consider the special case where p(x) — 0 and q ( 7 ;) — ]. Show that the function 0 ( 71 ) — c\ sin 7 , 4 - C 2 COS x ' s ' n the 
nullspace of L for all real values of c\ and cj- 


23. (For Readers Who Have Studied Calculus) 

Let D.P-n _» P n be the differentiation operator D( p) = p\ Show that the matrix for D with respect to the basis 

B = jl, x, x 2 , ..., j is 

'OIOO - 0" 

0 0 2 0 - 0 

0 0 0 3 — 0 

0 0 0 0 ■■■ n 

0 0 0 0 -0 


24. (For Readers Who Have Studied Calculus) 

It can be shown that for any real number c, the vectors 


form a basis for p. . Find the matrix for the differentiation operator of Exercise 23 with respect to this basis. 


25. (For Readers Who Have Studied Calculus) 

Let J\P n -> P n 1 be the integration transformation defined by 

f-X 

l/( 1>)= / (tf(H - b a„x n )dx = a$x I ^-x 2 f -+ x H+1 

J 0 l n V \ 

where p = -\- a\x \ —| a^x™. Find the matrix for T with respect to the standard bases for p n and P n +\- 
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Chapter 8 



Technology Exercises 


The following exercise is designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear 
algebra capabilities. For this exercise you will need to read the relevant documentation for the particular utility you are using. The 
goal of this exercise is to provide you with a basic proficiency with your technology utility. Once you have mastered the 
techniques in this exercise, you will be able to use your technology utility to solve many of the problems in the regular exercise 
sets. 


Section 8.3 

Tl. (Transition Matrices) Use your technology utility to verify Formula (5). 

Section 8.5 

Tl. (Similarity Invariants) Choose a nonzero 3x3 matrix A and an invertible 3x3 matrix P. Compute p -1 j^p and confirm 
the statements in Table 1. 
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9 

CHAPTER 


Additional Topics 


NTRODUCTION In this chapter we shall see how some of the topics that we have studied in earlier chapters can be 
applied to other areas of mathematics, such as differential equations, analytic geometry, curve fitting, and Fourier series. The 
chapter concludes by returning once again to the fundamental problem of solving systems of linear equations Ax = \> This 
time we solve a system not by another elimination procedure but by factoring the coefficient matrix into two different triangular 
matrices. This is the method that is generally used in computer programs for solving linear systems in real-world applications. 
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9.1 

APPLICATION TO 

DIFFERENTIAL 

EQUATIONS 


Many laws of physics, chemistry, biology, engineering, and economics are 
described in terms of differential equations—that is, equations involving 
functions and their derivatives. The purpose of this section is to illustrate 
one way in which linear algebra can be applied to certain systems of 
differential equations. The scope of this section is narrow, but it illustrates 
an important area of application of linear algebra. 


Terminology 

One of the simplest differential equations is 

y ,=a y (i) 

where y = J (x) is an unknown function to be determined, y ! — dyidx is its derivative, and a is a constant. Like most 
differential equations, 1 has infinitely many solutions; they are the functions of the form 

y =ceax ( 2 ) 

where c is an arbitrary constant. Each function of this form is a solution of y 1 — ay since 

y r — cae ax — ay 

Conversely, every solution of y f = ay must be a function of the form cs ax (Exercise 5), so 2 describes all solutions of 
y = ay. We call 2 the general solution of y ! = ay. 

Sometimes the physical problem that generates a differential equation imposes some added conditions that enable us to 
isolate one particular solution from the general solution. For example, if we require that the solution of y 1 = ay satisfy the 
added condition 


y(0) = 3 


(3) 


that is, y = 3 when * = Q, then on substituting these values in the general solution y = ce ax we obtain a value for c —namely, 
3 = ce° = c- Thus 

y = 3e ax 

is the only solution of y = ay that satisfies the added condition. A condition such as 3, which specifies the value of the 
solution at a point, is called an initial condition, and the problem of solving a differential equation subject to an initial 
condition is called an initial-value problem. 


Linear Systems of First-Order Equations 

In this section we will be concerned with solving systems of differential equations having the form 

y[ =a\\y\ +^ 12^2 + -" 4- 

y ! 'l =fl2Lyi + ^22^2 + - + <32 ny» 

y ! 4 = Gn\y\ 4 a rQy2 4 - 4 

where y j = f i (*), y 2 — /20O> — f n(x) aro f unc ti° ns to be determined, and the flyy's are constants. In matrix 

notation, 4 can be written as 





or, more briefly. 


A i 

Ai 


'an 

a 21 

3 12 ■ 

^22 ' 

,J a ln 

" a 2n 

>r 

72 

An 


fl H l 

a n2 ‘ 


y n 


v 


= j4y 


Solution of a System with Initial Conditions 


(a) Write the following system in matrix form: 


y[ - 3yi 

A = ~ 2 72 
A = 5 73 


(b) Solve the system. 


(c) Find a solution of the system that satisfies the initial conditions y j (0) = 1, (0) = 4, and y 3 (0) = — 2- 


Solution (a) 


or 


Ai 


o 

o 

on 

'71' 

A 

= 

0-2 0 

72 



0 0 5 

73 

y 3 





v 




3 

0 

0 



0 

0 

5 


y 


(5) 


Solution (b) 

Because each equation involves only one unknown function, we can solve the equations individually. From 2, we obtain 

3x 

y\ = n« 

y2 = c 3 e~ 2x 
5x 

y 3 - c 3 e 


or, in matrix notation, 


















Solution (c) 

From the given initial conditions, we obtain 


1 = 7l(0) =cie° = ci 

4 =72(0) = C2 e ° — c 2 
- 2 = 73 ( 0 ) = c 3 e°=c 3 

so the solution satisfying the initial conditions is 

yi=e 3 \ y 2 = 4e~ 2 \ y 3 =-2 e 3 ’ 

or, in matrix notation, 



>r 


1 

on 

y = 

72 

= 

4e- 2 * 


73 


-2e 5x _ 


The system in the preceding example is easy to solve because each equation involves only one unknown function, and this is 
the case because the matrix of coefficients for the system in 5 is diagonal. But how do we handle a system y r = j4v in which 
the matrix A is not diagonal? The idea is simple: Try to make a substitution for y that will yield a new system with a diagonal 
coefficient matrix; solve this new simpler system, and then use this solution to determine the solution of the original system. 


The kind of substitution we have in mind is 

71 = P liai bp 12^2 H- bp\ n u n 

72 = P2\u\ + P22U2 4-F P2n^n 


or, in matrix notation, 


Pn=Pn 1“1+ PyQP2 + 4- Pnn^n 


71 ‘ 


>11 

P 12 ■ 

" Pin' 

"«l" 



72 


P21 

P22 ' 

" P2n 

u 2 

or, more briefly. 

y = Til 

7h 


Pn\ 

PyQ. • 

" Pnn 

U Yl 


(7) 


In this substitution, the p^-% are constants to be determined in such a way that the new system involving the unknown 
functions u\,u 3 , has a diagonal coefficient matrix. We leave it for the reader to differentiate each equation in 7 and 

deduce 

y' = Fii' 

If we make the substitutions y = Pa and y f = p n ! in the original system 

y' = j 4 v 

and if we assume P to be invertible, then we obtain 

Pi i' = j 4 (Pu) 
or 

u' = (^ _1 ^)u or u'=.Dii 

where p _ p~l j^p. The choice for P is now clear; if we want the new coefficient matrix D to be diagonal, we must choose P 
















to be a matrix that diagonalizes A. 


Solution by Diagonalization 

The preceding discussion suggests the following procedure for solving a system y' = Ay with a diagonalizable coefficient 
matrix A. 

Step 1 . Find a matrix P that diagonalizes A. 

Step 2. Make the substitutions v = Pn and y r = Pn ! to obtain a new “diagonal system” u / = Du, where p) = p~^AP- 
Step 3. Solve u r = Du. 

Step 4. Determine y from the equation y = Ai. 


Solution Using Diagonalization 


(a) Solve the system 


y[= y i + 72 
72 =^y\ - 2 y 2 

(b) Find the solution that satisfies the initial conditions y j ( 0 ) = 1 , 72 ( 0 ) = 6 - 


Solution (a) 

The coefficient matrix for the system is 




As discussed in Section 7.2, A will be diagonalized by any matrix P whose columns are linearly independent eigenvectors of 
A. Since 


det(A/ — A) 


A- 1 
-4 


- 1 

A+ 2 


= A 2 F A — 6 = (A + 3) (A — 2) 


the eigenvalues of A are A = 2, A = — 3- By definition, 

"*r 

X_ |/ 2 _ 

is an eigenvector of A corresponding to A if and only if x is a nontrivial solution of (A/ — A)x = 0—that is, of 










If A = 2, this system becomes 


A— 1 -1 

'*l' 


0 

. “ 4 A + 2 _ 

/ 2 _ 


_ 0 _ 


1 

- r 

■*r 


'O' 

_ —4 

4_ 

_* 2 _ 


_o_ 


■*r 


~i 

— + 

T 

*2 


t 

— L 

_l_ 


Solving this system yields ^ — t, — £, so 


Thus 

l'l = 

is a basis for the eigenspace corresponding to \ = 2- Similarly, the reader can show that 

P 2 = 

is a basis for the eigenspace corresponding to \ = _ 3 . Thus 


P = 


1 -i 
1 1 


diagonalizes A, and 

Therefore, the substitution 
yields the new “diagonal system” 

From 2 the solution of this system is 


D = P~ l AP = 


2 0 
0 -3 


y = Pu and y f — Pii f 


u ; = ZHi = 




2 0 
0 -3 

2x 


u or 


= 2u\ 

^2= - 3ii2 


u 2 =c 2 e 

so the equation y = Pu yields, as the solution fory, 


-3* 


or u = 


c\e 


2x 


c 2 e 


—3x 


v = 


>r 


’l 

_ l" 

4 




. „2x 1 — 2x 

c \ s ~J C 2^ 

A2 


_1 

1_ 


c 2e - 3 ' 


1 

1 — <■ 

to 

4 

to 

1 

LO 

1_ 


or 


y\=c\e 2s - ±c-p 3 * 
y 2 = c l € 2x + c 2 e~ 2 * 


(8) 


Solution (b) 

If we substitute the given initial conditions in 8 , we obtain 

ci-^2=l 

C] + C ') = 6 

Solving this system, we obtain cj = 2> c 2 = 4 so from 8 , the solution satisfying the initial conditions is 








































We have assumed in this section that the coefficient matrix of v' = Ay is diagonalizable. If this is not the case, other methods 
must be used to solve the system. Such methods are discussed in more advanced texts. 


Exercise Set 9.1 



Click here for Just Ask! 


1 . 

(a) Solve the system 


y [ = 71+472 
y*2 ~ 2 yi + 3 y 2 

(b) Find the solution that satisfies the initial conditions y j ( 0 ) = 0 , y 2 (0) = 0- 


2 . 

(a) Solve the system 

y[= yi + 3y2 

y2= 4 y\ + 5 y2 

(b) Find the solution that satisfies the conditions y j ( 0 ) = 2, y f 2 (0) = 1- 


3. 

(a) Solve the system 

y[ = 4y i h y 3 
y ! 2 = ~ 2 yi+y2 

y 2 = -2yi H 73 

(b) Find the solution that satisfies the initial conditions y j ( 0 ) = — 1, y 2 (0) = L 73 (G) = 0- 


4 . 


Solve the system 




y [ = 471 H 2^2-1-273 

y’ 2 = 2y\ 4 4 y 2 4 - 2y 3 
73 = 2^i 4- 2^2 + 473 


Show that every solution of y ! = ay has the form y — ce ax . 

5. 

Hint Let y — f (*) be a solution of the equation, and show that / (x)e~ ax is constant. 


Show that if A is diagonalizable and 

6 * [>r 

72 

y = 

7h 

satisfies y 1 ' = Ay, then each y,- is a linear combination of e ^ 2 x , where Ap \ 2 , ..., A H are the eigenvalues of 

A. 

It is possible to solve a single differential equation by expressing the equation as a system and then using the method of 
7- this section. For the differential equation y !! —y ! — 6y = 0, show that the substitutions —y and y 2 = y ! lead to the 
system 

71 —72 

72 = 6 71 +72 

Solve this system and then solve the original differential equation. 

Use the procedure in Exercise 7 to solve y n +y ! — 12y = 0. 

8 . 


9. 


Discuss: How can the procedure in Exercise 7 be used to solve y >!> — 6y !! 4- 11 y f — by = 0? Carry out your ideas. 


Discussion 

Discovery 

lUi 

(a) By rewriting 8 in matrix form, show that the solution of the system in Example 2 can 
be expressed as 

l" 

4 

1 _ 

This is called the general solution of the system. 

(b) Note that in part (a), the vector in the first term is an eigenvector corresponding to the 
eigenvalue Ai = 2 and the vector in the second term is an eigenvector corresponding t( 
the eigenvalue \ 2 = _ 3 . This is a special case of the following general result: 


2x 

"f 

1 —-'X 

y — c i e 


+ c 2 e 


THEOREM 










If the coefficient matrix A of the system y 1 = ^y in 4 is diagonalizable, then the 
general solution of the system can be expressed as 

y = + c 2 e^*x 2 + —V c H 

where Ai, X2, ■■■, A„ are the eigenvalues of A, and is an 
eigenvector of A corresponding to a, . 

Prove this result by tracing through the four-step procedure discussed in the section 
with 


D = 


Ai 0 - 0 

0 A 2 ... 0 


0 0 - a h 


and P= [xi|x 2 |.|x H ] 


Consider the system of differential equations y / = Ay where A is a 2 x 2 matrix. For what 
H* values of an, fli 2 , a 2 j, o 22 do the component solutions y 1 (t), y 2 (0 tend to zero as t —> 00 
? In particular, what must be true about the determinant and the trace of A for this to happen? 

Solve the nondiagonalizable system y ! j —y \ Ay 2 ^ y f 2 —y2- 


Use diagonalization to solve the system y ! ^ = 2y\ f y2 A U y ! 2 = y\ )- 2y 2 -|- 2t by first 
13. writing it in the form y' = j4y + f . Note the presence of a forcing function in each equation. 

Use diagonalization to solve the system y / 1 =yi-l-y 2 -l-e f .y / 2 =yi—y 2 + e _f by first 
1^* writing it in the form y 1 — Ay A f. Note the presence of a forcing function in each equation. 
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Q 2 ln Section 4.2 we studied some of the geometric properties of linear operators 

r , cr . MC _ rDV _ on R 2 and R 3 . In this section we shall study linear operators on R 2 in a little 

GEOMETRY OF LINEAR 

OPERATORS ON R 2 


Vectors or Points 

If 7; R 2 -♦ R 2 is the matrix operator whose standard matrix is 

A = 

then 

T 


a b 
c d 


/pi 


a b 

~x~ 


ax 

1 by' 


r 

c d 

y 


cx 

1 dy 


( 1 ) 


There are two equally good geometric interpretations of this formula. We may view the entries in the matrices 

ax f by 


and 


cx + dy 


either as components of vectors or as coordinates of points. With the first interpretation, T maps arrows to arrows, and with the 
second, points to points (Figure 9.2.1). The choice is a matter of taste. 



(a) T maps vectors 1o vectors 


v • (<m ■+■ by, c.v + dy 1 


i u, M 


(h) 7 maps points Lo points 
Figure 9.2.1 


In this section we shall view linear operators on R 2 as mapping points to points. One useful device for visualizing the behavior of a 
linear operator is to observe its effect on the points of simple figures in the plane. For example, Table 1 shows the effect of some 
basic linear operators on a unit square that has been partially colored. 


Table 1 


























In Section 4.2 we discussed reflections, projections, rotations, contractions, and dilations of p l . We shall now consider some other 
basic linear operators on p}. 


Compressions and Expansions 


If the x-coordinate of each point in the plane is multiplied by a positive constant k, then the effect is to compress or expand each 
plane figure in the x-direction. If 0 < k < L the result is a compression, and if k > 1, it is an expansion (Figure 9.2.2). We call such 
an operator a compression (or an expansion ) in the x-direction with factor k. Similarly, if the y-coordinate of each point is 
multiplied by a positive constant k, we obtain a compression (or expansion ) in the y-direction with factor k. It can be shown that 
compressions and expansions along the coordinate axes are linear transformations. 






'■ 



L 

X 




ih'\ i Compression) k = J 



luj (Unit square) 
Figure 9.2.2 


(c) {Expansion} t = 2 




























































If 7 : pf _* p} is a compression or expansion in the x-direction with factor k, then 

0 
1 

so the standard matrix for T is 

~k O' 

0 1 


T(<n) = T 


1 

_ 

k 

A 

r 

0 


T(*2) = T 


( 0 

_ 

1 

r 


Similarly, the standard matrix for a compression or expansion in the y-direction is 


1 0 
0 k 


Operating with Diagonal Matrices 


Suppose that the xy-plane first is compressed or expanded by a factor of ^ in the x-direction and then is compressed or expanded 
by a factor of £ 2 i n the y-direction. Find a single matrix operator that performs both operations. 


>1 o' 


'1 0' 

_ 0 i_ 


_° * 2 _ 


Solution 

The standard matrices for the two operations are 


x — coitipi’essioitjespajtsioni>' — compi’essioitiespajision) 

Thus the standard matrix for the composition of the x-operation followed by the y-operation is 

A = 

This shows that multiplication by a diagonal 2x2 matrix compresses or expands the plane in the x-direction and also in the 
y-direction. In the special case where k\ and £ 2 are the same, say k\=k 2 = k note that 2 simplifies to 

'k O' 


'1 0 ' 

k\ 

o' 


*1 

0 ' 

_0 k 2 _ 

_ 0 

1_ 


0 

h 


( 2 ) 


A = 


0 * 


which is a contraction or a dilation (Table 8 of Section 4.2). 


Shears 

A shear in the x-direction with factor k is a transformation that moves each point ( x , y") parallel to the x-axis by an amount ky to 
the new position ( x 4 ky, 7). Under such a transformation, points on the x-axis are unmoved since y = Q. However, as we 
progress away from the x-axis, the magnitude of y increases, so points farther from the x-axis move a greater distance than those 
closer (Figure 9.2.3). 































{ft) Shear tn .^direction 
with factor i > 0 



If’} Shear in r-directioii 
with factor k < 0 


Figure 9.2.3 


A shear in the y-direction with factor k is a transformation that moves each point (x, y) parallel to the y-axis by an amount kx to 
the new position (x, y f kx) - Under such a transformation, points on the y-axis remain fixed, and points farther from the y-axis 
move a greater distance than those that are closer. 


It can be shown that shears are linear transformations. If r : R l 


so the standard matrix for T is 


T(*x) = T 



g} is a shear with factor k in the x-direction, then 


n* 2 )=T 



1 k 
0 1 


Similarly, the standard matrix for a shear in the y-direction with factor k is 


1 0 
k 1 


Remark Multiplication by the 2 x 2 identity matrix is the identity operator on gf-. This operator can be viewed as a rotation 
through 0°, or as a shear along either axis with k = 0> or as a compression or expansion along either axis with factor k = 1 • 


























Finding Matrix Transformations 


(a) Find a matrix transformation from p} to p} that first shears by a factor of 2 in the x-direction and then reflects about y = x. 


(b) Find a matrix transformation from p} to p} that first reflects about y = x and then shears by a factor of 2 in the x-direction. 


Solution (a) 

The standard matrix for the shear is 

and for the reflection is 


Ai = 


A 2 = 


1 2 
0 1 


0 1 
1 0 


Thus the standard matrix for the shear followed by the reflection is 

A 2 A 1 = 


Solution (b) 

The reflection followed by the shear is represented by 

2M2 = 


1- 

o 

'1 2' 


1 

o 

1_ 

_1 

O 


_ 1 2_ 


'1 2' 

i 

o 


1 

1_ 

-1 

O 

_1 

1- 

o 

1_ 


1 

o 

1—1 

_1 


In the last example, note that A\A 2 * A 2 Ay so the effect of shearing and then reflecting is different from the effect of reflecting and 
then shearing. This is illustrated geometrically in Figure 9.2.4, where we show the effects of the transformations on a unit square. 





















Figure 9.2.4 


EXAMPLE 3 Transformations Using Elementary Matrices 


Show that if j- p} _> p} is multiplication by an elementary matrix, then the transformation is one of the following: 

(a) a shear along a coordinate axis 


(b) a reflection about y = x 


(c) a compression along a coordinate axis 

(d) an expansion along a coordinate axis 

(e) a reflection about a coordinate axis 


(f) a compression or expansion along a coordinate axis followed by a reflection about a coordinate axis. 


Solution 







































Because a 2 x 2 elementary matrix results from performing a single elementary row operation on the 2x2 identity matrix, it must 
have one of the following forms (verify): 


1 

o 

_1 


1 


o 

1—«■ 

_1 


~k O' 


l 

o 

k 1_ 


1 

o 


1 

o 

_1 


_° 


_0 k_ 


The first two matrices represent shears along coordinate axes; the third represents a reflection about y = x. If k > 0- the last two 
matrices represent compressions or expansions along coordinate axes, depending on whether 0 < k < 1 or k > 1. If k < 0> and if we 
express k in the form £ = _ where ^ > 0, then the last two matrices can be written as 

*1 O' 


~k O' 


l 

1 

o 

_i 


i 

o 

1 

1 _ 

l 

O 


0 1 


1 

o 


1 o' 


'1 O' 


'1 O' 

'1 0 ' 

0 k_ 


0 -*i 


_0 - 1 _ 

0 *1 


(4) 


Since > 0» the product in 3 represents a compression or expansion along the x-axis followed by a reflection about the y-axis, and 
4 represents a compression or expansion along the y-axis followed by a reflection about the x-axis. In the case where fc = _ l, 
transformations 3 and 4 are simply reflections about the y-axis and x-axis, respectively. 

♦ 


Reflections, rotations, compressions, expansions, and shears are all one-to-one linear operators. This is evident geometrically, 
since all of those operators map distinct points into distinct points. This can also be checked algebraically by verifying that the 
standard matrices for those operators are invertible. 


EXAM A Transformation and Its Inverse 


It is intuitively clear that if we compress the xy-plane by a factor of in the y-direction, then we must expand the xy-plane by a 
factor of 2 in the y-direction to move each point back to its original position. This is indeed the case, since 

1 O' 


A = 


0 i 


represents a compression of factor -i in the y-direction, and 


= 


1 0 
0 2 


is an expansion of factor 2 in the y-direction. 


Geometric Properties of Linear Operators on R 2 

We conclude this section with two theorems that provide some insight into the geometric properties of linear operators on g}. 


THEOREM 9.2.1 


If T\ R 2 — >R 2 is multiplication by an invertible matrix A, then the geometric effect of T is the same as an appropriate 
succession of shears, compressions, expansions, and reflections. 


Proof Since A is invertible, it can be reduced to the identity by a finite sequence of elementary row operations. An elementary 
row operation can be performed by multiplying on the left by an elementary matrix, and so there exist elementary matrices E\> By 




































• Efc such that 


E^—E 2 E\A = l 

Solving for A yields 
or, equivalently, 


This equation expresses A as a product of elementary matrices (since the inverse of an elementary 
matrix is also elementary by Theorem 1.5.2). The result now follows from Example 3. 


EXAMPLE 5 Geometric Effect of Multiplication by a Matrix 

Assuming that and i : - 2 are positive, express the diagonal matrix 

'*i 0 ' 

^ = [o k 2 

as a product of elementary matrices, and describe the geometric effect of multiplication by A in terms of compressions and 
expansions. 


Solution 


From Example 1 we have 


>1 0 ' 


'1 0 ' 

■*1 0" 

o 

to 

1 _ 


0 k 2 

_0 1_ 


which shows that multiplication by A has the geometric effect of compressing or expanding by a factor of in the x-direction and 
then compressing or expanding by a factor of in the y-direction. 


EXAMPLE 6 Analyzing the Geometric Effect of a Matrix Operator 


Express 


as a product of elementary matrices, and then describe the geometric effect of multiplication by A in terms of shears, compressions, 
expansions, and reflections. 


Solution 


A can be reduced to I as follows: 
















T T T 

Add — 3 times Multiply the Add — 2 times 

the first row second row the second row 

to the second. by — ^ to the first. 

The three successive row operations can be performed by multiplying on the left successively by 


1 0 
-3 1 


S 2 = 




£l = 

Inverting these matrices and using 5 yields 

Reading from right to left and noting that 

it follows that the effect of multiplying by A is equivalent to 
shearing by a factor of 2 in the x-direction, 

2. then expanding by a factor of 2 in the y-direction, 


1 0 

0 -k 

2 


e 3 = 


i 


1 _ 

'1 o ' 

'1 

O ' 

'1 2 ' 


3 1 

0 

-2 

0 1 


'1 O ' 


'1 

O ' 

'1 O ' 

_° — 2 _ 


_0 

- 1 _ 

_0 2 _ 


3, then reflecting about the x-axis. 


then shearing by a factor of 3 in the y-direction. 


The proofs for parts of the following theorem are discussed in the exercises. 

THEOREM 9.2.2 

Images of Lines 

If T\ R 2 — >R 2 is multiplication by an invertible matrix , then 

(a) The image of a straight line is a straight line. 

(b) The image of a straight line through the origin is a straight line through the origin. 

(c) The images of parallel straight lines are parallel straight lines. 

(d) The image of the line segment joining points P and Q is the line segment joining the images of P and Q. 






























(e) The images of three points lie on a line if and only if the points themselves lie on some line. 


Remark It follows from parts (c), (d). and (e) that multiplication by an invertible 2x2 matrix A maps triangles into triangles and 
parallelograms into parallelograms. 


Image of a Square 


The square with vertices /^(O, 0)> P 2 (1, 0)- 1, 1), and P^f), 1) is called the unit square. Sketch the image of the unit square 

under multiplication by 



Solution 

Since 


' - 1 2' 

'O' 


'O' 


' - 1 2' 

t 


2 -1_ 

_0_ 


_0_ 


2 -1_ 

_0_ 


' - 1 2' 

'O' 



2' 


' - 1 2' 

t 

r 

2 -!_ 

_1_ 


— 

1_ 


2 -1_ 

_i_ 



the image of the square is a parallelogram with vertices (0, 0), (-1, 2), (2, -1), and (1, 1) (Figure 9.2.5). 









































Image of a Line 


According to Theorem 2, the invertible matrix 


maps the line y = 2x \ 1 into another line. Find its equation. 


Solution 

Let be a point on the line y = 2x \ 1, and let (x ! ,y ! ) be its image under multiplication by A. Then 


V 


'3 f 

~x~ 


~x~ 


'3 r 

-1 

V 


1 - r 

V 

_/ 


.2 1. 

y 

and 

y 


.2 1 . 


/_ 


. -2 3_ 

/_ 


so 


x = X - 7 

7 = - 2*' 4 3/ 


Substituting in y = 2x 4- 1 yields 


— 2x } + 3 y ! = 2{x’ — y 1 ) + 1 or, equivalently. 


j 


y=i/4i 


Thus {x ! ,y ! ) satisfies 


which is the equation we want. 


^=F+j 


♦ 


Exercise Set 9.2 




Click here for Just Ask! 


1 . 


Find the standard matrix for the linear operator f P? _> p? that maps a point (x, 7 ) into (see the accompanying figure) 


(a) its reflection about the line y = — x 


(b) its reflection through the origin 


(c) its orthogonal projection on the x-axis 


(d) its orthogonal projection on the v-axis 
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ill) 

Figure Ex-1 

For each part of Exercise 1, use the matrix you have obtained to compute T(2, 1). Check your answers geometrically by plotting 
2. the points (2, 1) and T(2, 1). 


Find the standard matrix for the linear operator j- p} _> p} that maps a point (x, y , z) into 


(a) its reflection through the xy-plane 


(b) its reflection through the plane 


(c) its reflection through the yz-plane 


For each part of Exercise 3, use the matrix you have obtained to compute T (1, 1, 1). Check your answers geometrically by 
4. sketching the vectors (1, 1, 1) and T (1, 1, 1). 











Find the standard matrix for the linear operator T B? _* R ~~' that 

5. 


(a) rotates each vector 90° counterclockwise about the z-axis (looking along the positive z-axis toward the origin) 


(b) rotates each vector 90° counterclockwise about the x-axis (looking along the positive x-axis toward the origin) 


(c) rotates each vector 90° counterclockwise about the y-axis (looking along the positive y-axis toward the origin) 


Sketch the image of the rectangle with vertices (0, 0), (1, 0), (1, 2), and (0, 2) under 

6 . 

(a) a reflection about the x-axis 


(b) a reflection about the y-axis 


(c) a compression of factor k = j in the y-direction 


(d) an expansion of factor k = 2 in the x-direction 


(e) a shear of factor k = 3 in the x-direction 


(f) a shear of factor k = 2 in the y-direction 


Sketch the image of the square with vertices (0, 0), (1, 0), (0, 1), and (1,1) under multiplication by 



Find the matrix that rotates a point (x v) about the origin through 

8 . ' 

(a) 45° 

(b) 90° 

(c) 180° 

(d) 270° 

(e) -30° 


9. 


Find the matrix that shears by 



(a) a factor of k = 4 in the y-direction 


(b) a factor of k = — 2 in the ^-direction 


Find the matrix that compresses or expands by 

10 . 

(a) a factor of ^ in the y-direction 

(b) a factor of 6 in the ^-direction 


In each part, describe the geometric effect of multiplication by the given matrix. 

11 . 


(a) 

(b) 

(c) 


3 0 
0 1 


1 0 
0 -5 


1 4 
0 1 


Express the matrix as a product of elementary matrices, and then describe the effect of multiplication by the given matrix 
12. terms of compressions, expansions, reflections, and shears. 


(a) 

(b) 

(c) 

(d) 


2 0 
0 3 


1 4 

2 9 


0 -2 
4 0 


1 -3 

4 6 


13. 


In each part, find a single matrix that performs the indicated succession of operations: 



(a) compresses by a factor of ^ in the x-direction, then expands by a factor of 5 in the y-direction 


(b) expands by a factor of 5 in the y-direction, then shears by a factor of 2 in the y-direction 

(c) reflects about y = x, then rotates through an angle of 180° about the origin 


In each part, find a single matrix that performs the indicated succession of operations: 

14. 

(a) reflects about the y-axis, then expands by a factor of 5 in the x-direction, and then reflects about y = x 

(b) rotates through 30° about the origin, then shears by a factor of -2 in the y-direction, and then expands by a factor of 3 
in the y-direction 


By matrix inversion, show the following: 

15. 

(a) The inverse transformation for a reflection about y = x is a reflection about y = x. 


(b) The inverse transformation for a compression along an axis is an expansion along that axis. 

(c) The inverse transformation for a reflection about a coordinate axis is a reflection about that axis. 


(d) The inverse transformation for a shear along a coordinate axis is a shear along that axis. 


Find the equation of the image of the line y = _ % 4 3 under multiplication by 



In parts (a) through (e), find the equation of the image of the line y = 2x under 

17. “ ^ 


(a) a shear of factor 3 in the x-direction 


(b) a compression of factor in the y-direction 


(c) a reflection about y = x 


(d) a reflection about the y-axis 


(e) a rotation of 60° about the origin 



Find the matrix for a shear in the x-direction that transforms the triangle with vertices (0, 0), (2, 1), and (3, 0) into a right 
18. triangle with the right angle at the origin. 


19. 

(a) Show that multiplication by 


maps every point in the plane onto the line y = 2x- 

(b) It follows from part (a) that the noncollinear points (1,0), (0, 1), (-1, 0) are mapped on a line. Does this violate part (e) 
of Theorem 2? 


Prove part (a) of Theorem 2. 

20 . 

Hint A line in the plane has an equation of the form Ax V By \ C = 0, where A and B are not both zero. Use the method of 
Example 8 to show that the image of this line under multiplication by the invertible matrix 


a b 
c d 

has the equation A'x a By a C= 0, where 

A' = {dA-cB)i {ad-bc) and B ! = (-bA + aB) I (ad-be) 

Then show that A s and B r are not both zero to conclude that the image is a line. 


Use the hint in Exercise 20 to prove parts ( b ) and (c) of Theorem 2. 

21 . 


In each part, find the standard matrix for the linear operator T. B? _> R ~~' described by the accompanying figure. 





In r} the shear in the xy-direction with factor k is the linear transformation that moves each point y r z ) parallel to the ?; 
-plane to the new position + ky, y A kz, z) ■ (See the accompanying figure.) 


(a) Find the standard matrix for the shear in the xy-direction with factor k. 


(b) How would you define the shear in the ^-direction with factor k and the shear in the yz-direction with factor kl Fin< 
the standard matrices for these linear transformations. 















—I l_ 

V 7 

A 



Figure Ex-23 


In each part, find as many linearly independent eigenvectors as you can by inspection (by visualizing the geometric effect of 
24. the transformation on p}). For each of your eigenvectors find the corresponding eigenvalue by inspection; then check your 
results by computing the eigenvalues and bases for the eigenspaces from the standard matrix for the transformation. 


(a) reflection about the x-axis 


(b) reflection about the y-axis 


(c) reflection about y = x 


(d) shear in the x-direction with factor k 


(e) shear in the y-direction with factor k 


(f) rotation through the angle 0 
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9.3 

LEAST SQUARES 
FITTING TO DATA 


In this section we shall use results about orthogonal projections in inner 
product spaces to obtain a technique for fitting a line or other polynomial 
curve to a set of experimentally determined points in the plane. 


Fitting a Curve to Data 

A common problem in experimental work is to obtain a mathematical relationship y = f (*) between two variables x and y 
by “fitting” a curve to points in the plane corresponding to various experimentally determined values of x and y, say 

On the basis of theoretical considerations or simply by the pattern of the points, one decides on the general form of the curve 
y = f (x) to be fitted. Some possibilities are (Figure 9.3.1) 

(a) A straight line: y =a + bx 

(b) A quadratic polynomial: y — a 4- bx + cx 1 

(c) A cubic polynomial: y = a + bx + cx 2 + dx 3 





( a) y = it + ft_r 




Figure 9.3.1 

Because the points are obtained experimentally, there is usually some measurement “error” in the data, making it impossible 
to find a curve of the desired form that passes through all the points. Thus, the idea is to choose the curve (by determining its 
coefficients) that “best” fits the data. We begin with the simplest and most common case: fitting a straight line to the data 
points. 

Least Squares Fit of a Straight Line 

Suppose we want to fit a straight line y — a \ bx to the experimentally determined points 

If the data points were collinear, the line would pass through all n points, and so the unknown coefficients a and b would 
satisfy 

y\=a-\- 
72 = a \-bx 2 

y n =a f bxyi 

We can write this system in matrix form as 


1 XI 



>r 

1 *2 


~a~ 


yi 

: : 


b_ 


: 

1 x H 



yn 














or, more compactly, as 


where 


Mv = y 



>r 


'l 

*1 

y = 

72 

M = 

1 

*2 


7h 


1 

x n 



( 1 ) 


( 2 ) 


If the data points are not collinear, then it is impossible to find coefficients a and b that satisfy system 1 exactly; that is, the 
system is inconsistent. In this case we shall look for a least squares solution 


v = v — 


* * 


We call a line y — a -f b x whose coefficients come from a least squares solution a regression line or a least squares 
straight line fit to the data. To explain this terminology, recall that a least squares solution of 1 minimizes 

||y — Mv || 

If we express the square of 3 in terms of components, we obtain 

||y — Mv|p = Oi -a-bx i ) 2 -I- O 2 - a -bxf) 2 + ‘"4 (y„ -a -bx n ) 2 

If we now let 


then 4 can be written as 


d\ = \y\-a-bx\\, ^2 = 172 — ---> d yi = \y yi -a-bx yi \ 


|y — Mv|p = dj + dj+~ + di 


(3) 


(4) 


(5) 


As illustrated in Figure 9.3.2, ^. can be interpreted as the vertical distance between the line y = a 1 bx and the data point 
(xj, y,-). This distance is a measure of the “error” at the point (x,-, jy 3 ) resulting from the inexact fit of y = a 4 . bx to the data 
points. The assumption is that the x,- are known exactly and that all the error is in the measurement of the y,-. We model the 
error in the y,- as an additive error—that is, the measured y, is equal to a \ bxj I dj f° r some unknown error dr Since 3 and 
5 are minimized by the same vector v *, the least squares straight line fit minimizes the sum of the squares of the estimated 
errors dr hence the name least squares straight line fit. 



Figure 9.3.2 

dj measures the vertical error in the least squares straight line. 


Normal Equations 

Recall from Theorem 6.4.2 that the least squares solutions of 1 can be obtained by solving the associated normal system 
















M T My=M T y 

the equations of which are called the normal equations. 

In the exercises it will be shown that the column vectors of M are linearly independent if and only if the n data points do not 
lie on a vertical line in the xy-plane. In this case it follows from Theorem 6.4.4 that the least squares solution is unique and is 
given by 

v = (M r M) M r y 

In summary, we have the following theorem. 


THEOREM 9.3.1 


Least Squares Solution 

Let (x i, yi)> (*2> yi)’---’ (*h> yn) ^ e a set oft™ 0 or more data points, not all lying on a vertical line, and let 



1 xi 


>r 

M = 

1 x 2 

and y = 

yi 


1 x H 


yn 


Then there is a unique least squares straight line fit 

* * 

y = a 4- b x 

to the data points. Moreover, 


* 

V = 


is given by the formula 


* 7 " — 1 7 " 

V = (M T M) M J y 


which expresses the fact that v = v* is the unique solution of the normal equations 

M T Mv=M T y 


( 6 ) 

(7) 


Least Squares Line: Using Formula 6 


Find the least squares straight line fit to the four points (0, 1), (1, 3), (2, 4), and (3, 4). (See Figure 9.3.3.) 











Figure 9.3.3 


Solution 


We have 


M = 


1 0 
1 1 
1 2 
1 3 


M r M — 


4 6 
6 14 


* 

V = 


(M T M) 1 M T y =-L 


and (M 1 M) - 


10 






1 



1 

-3' 

'1 

1 1 f 

3 

_ 

'1.5' 

-3 

2_ 

_0 

1 2 3_ 

4 

4 


_ 1 _ 


so the desired line is y = 1.5 + 


7 -3 
3 2 


Spring Constant 


Hooke's law in physics states that the length x of a uniform spring is a linear function of the force y applied to it. If we write 
y = a 4- bx, then the coefficient b is called the spring constant. Suppose a particular unstretched spring has a measured length 
of 6.1 inches (i.e., x = 6.1 when y = 0). Forces of 2 pounds, 4 pounds, and 6 pounds are then applied to the spring, and the 
corresponding lengths are found to be 7.6 inches, 8.7 inches, and 10.4 inches (see Figure 9.3.4). Find the spring constant of 
this spring. 


? Lon^lh 

5 x 
2 


l orrc y 


x* 


ri 

0 

7 6 

2 

8.7 

4 

10.4 
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Figure 9.3.4 


Solution 










































We have 


M — 


'l 

6.1" 


"o' 

1 

7.6 

, y = 

2 

1 

3.7 


4 

1 10.4 


6 


and 


* 

V = 


-1 


= (M J M) M T y^ 


3.6 

1.4 


where the numerical values have been rounded to one decimal place. Thus the estimated value of the spring constant is 
£*=14 pounds/inch. 

Least Squares Fit of a Polynomial 

The technique described for fitting a straight line to data points generalizes easily to fitting a polynomial of any specified 
degree to data points. Let us attempt to fit a polynomial of fixed degree m 


m 


y = an + a i * 4- Y a m x 

to n points 

(*i,.yi), ( x%y (x M ,.y M ) 

Substituting these n values of x and y into 8 yields the n equations 

y l —ao + aixi H-h a m xf 

y 2 = ao + aiX 2 I- Y a m x™ 


( 8 ) 


or, in matrix form, 


where 


y n = an+a\x yi 1-1 -a m x- 


Mx = v 


m 


As before, the solutions of the normal equations 


M T Mx = M T y 


(9) 



>r 


1 

*1 

A ■ 

„ 

X \ 


" 


y — 

72 

M 

1 

x 2 

A ■ 

*2 

* v = 


(10) 


7h 


1 

x n 

A ■ 

A H 





determine the coefficients of the polynomial. The vector v minimizes 

||y - Mv|| 

Conditions that guarantee the invertibility of M T M are discussed in the exercises. If M T M is invertible, then the normal 
equations have a unique solution v = v *, which is given by 


r^m -1 *sT 


v = (M M) M y 


( 11 ) 
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On October 5, 1991 the Magellan spacecraft entered the atmosphere of Venus and transmitted the temperature T in 
kelvins (K) versus the altitude h in kilometers (km) until its signal was lost at an altitude of about 34 km. Discounting the 
initial erratic signal, the data strongly suggested a linear relationship, so a least squares straight line fit was used on the 
linear part of the data to obtain the equation 


7 = 737.5-8.125/! 

By setting h = 0 in this equation, the surface temperature of Venus was estimated at 737.5K- 


Fitting a Quadratic Curve to Data 


According to Newton's second law of motion, a body near the earth's surface falls vertically downward according to the 
equation 

s = s 0 H- vqM jg£ 2 (12) 


where 


s = vertical displacement downward relative to some fixed point 
sq = initial displacement at time t — 0 
vq — initial velocity at time t = 0 
g = acceleration of gravity at the earth's surface 

Suppose that a laboratory experiment is performed to evaluate g using this equation. A weight is released with unknown 
initial displacement and velocity, and at certain times the distances fallen relative to some fixed reference point are 
measured. In particular, suppose it is found that at times t = . 1, .2, .3, .4, and .5 seconds, the weight has fallen 
s = — 0.18, 0.31, 1.03, 2.48, and 3.73 feet, respectively, from the reference point. Find an approximate value of g using these 
data. 


Solution 


The mathematical problem is to fit a quadratic curve 









(13) 


s = a\} + a\t 


I a 2 t 2 


to the five data points: 

(.1, -0.18), (.2,0.31), (.3,1.03), (.4,2.48), 

With the appropriate adjustments in notation, the matrices M and y in 10 are 


(.5, 3.73) 


M = 


1 

tl 

t 2 

h 






1 

l 2 

2 


l 

.1 

.01 


"sf 


" -0.18" 



l 

.2 

.04 


s 2 


0.31 

1 


t 2 

— 

l 

.3 

.09 

? V = 

S3 

— 

1.03 

1 




l 

.4 

.16 


s 4 


2.48 

<4 

t{ 


l 

.5 

.25 


S5 


3.73 

1 

h 

t 2 

h 







Thus, from 11, 


* 

V = 


= (M T M) X M T V = 


-0.40 

0.35 

16.1 


From 12 and 13, we have a2 = l.g, so the estimated value of g is 

g — 2a2 — 2(16.1) = 32.2 feet / second 2 

If desired, we can also estimate the initial displacement and initial velocity of the weight: 

* 

s\~_\ = fig = —0.40 feet 
* 

vo = flj = 0.35 feet / second 

In Figure 9.3.5 we have plotted the five data points and the approximating polynomial. 



Figure 9.3.5 


Exercise Set 9.3 



Click here for Just Ask! 


Find the least squares straight line fit to the three points (0, 0), (1, 2), and (2, 7). 


1 . 


































Find the least squares straight line fit to the four points (0, 1), (2, 0), (3, 1), and (3, 2). 

2. 


Find the quadratic polynomial that best fits the four points (2, 0), (3, -10), (5, -48), and (6, -76). 

3. 


Find the cubic polynomial that best fits the five points (-1, -14), (0, -5), (1, -4), (2, 1), and (3, 22). 

4. 

Show that the matrix M in Equation 2 has linearly independent columns if and only if at least two of the numbers xi, x 2 , 

5. ... , x n are distinct. 

Show that the columns of the n x (m 4- 1) matrix M in Equation 10 are linearly independent if n > m and at least m + 1 of 
6* the numbers x\, X 2 , ..., x n are distinct. 

Hint A nonzero polynomial of degree m has at most m distinct roots. 


Let M be the matrix in Equation 10. Using Exercise 6, show that a sufficient condition for the matrix M t0 be 
invertible is that n > m and that at least m + 1 of the numbers x\, X 2 , ..., x n are distinct. 


The owner of a rapidly expanding business finds that for the first five months of the year the sales (in thousands) are $4.0, 
8. $4.4, $5.2, $6.4, and $8.0. The owner plots these figures on a graph and conjectures that for the rest of the year, the sales 
curve can be approximated by a quadratic polynomial. Find the least squares quadratic polynomial fit to the sales curve, 
and use it to project the sales for the twelfth month of the year. 


9. 


A corporation obtains the following data relating the number of sales representatives on its staff to annual sales: 

Number of Sales Representatives 5 10 15 20 25 30 


Annual Sales (millions) 3.4 4.3 5.2 6.1 7.2 8.3 

Explain how you might use least squares methods to estimate the annual sales with 45 representatives, and discuss the 
assumptions that you are making. (You need not perform the actual computations.) 

Find a curve of the form y = a + (b f x) that best fits the data points (1, 7), (3, 3), (6, 1) by making the substitution 
I®* x — 1 / x- Draw the curve and plot the data points in the same coordinate system. 


Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 




9.4 

APPROXIMATION 
PROBLEMS; FOURIER 
SERIES 


In this section we shall use results about orthogonal projections in inner 
product spaces to solve problems that involve approximating a given 
function by simpler functions. Such problems arise in a variety of 
engineering and scientific applications. 


Best Approximations 

All of the problems that we will study in this section will be special cases of the following general problem. 


Given a function/that is continuous on an interval [a, b], find the “best possible approximation” 
to/using only functions from a specified subspace Wof C\a,b~\- 

Here are some examples of such problems: 


(a) Find the best possible approximation to e x over [0, 1] by a polynomial of the form «q 4. a^x 4 0,2 x 2 - 

(b) Find the best possible approximation to sin xx over [-1, 1] by a function of the form _|_ a ^ e x \ ai< 3 ly ' 4 a^ x - 

(c) Find the best possible approximation to x over [0, 2~] by a function of the form 

AO fai sinx 4 £U1 2x 4 cos x 4 &2 cos 2x- 

hi the first example W is the subspace of CfO, 1 ] spanned by 1, x, and 7 }\ in the second example W is the subspace of 
C\ — 1, 11 spanned by 1, e x , e 2x , and e 2x ; and in the third example W is the subspace of C[0, 2 tt 1 spanned by 1, smx, sin 2x, 
cos x, and cos 2x- 

Measurements of Error 

To solve approximation problems of the preceding types, we must make the phrase “best approximation over \ a , & l” 
mathematically precise; to do this, we need a precise way of measuring the error that results when one continuous function is 
approximated by another over [a, £]. If we were concerned only with approximating f(x) at a single point xq, then the error 
at xo by an approximation g(^) would be simply 

error = 1/ (xo) - g(xo) I 

sometimes called the deviation between/and g at xo (Figure 9.4.1). However, we are concerned with approximation over the 
entire interval [a,b], not at a single point. Consequently, in one part of the interval an approximation gj to/may have 
smaller deviations from/than an approximation g 2 to/, and in another part of the interval it might be the other way around. 
How do we decide which is the better overall approximation? What we need is some way of measuring the overall error in an 
approximation g(x)- One possible measure of overall error is obtained by integrating the deviation \j (x) — g(x) I over the 
entire interval [a,by, that is, 


f b 

error = / |/ (x) - g(x) |dx 

Ja 


(i) 






_ 

■i. 


ti 


h 


Figure 9.4.1 

The deviation between/and g at xg. 


Geometrically, 1 is the area between the graphs of / ( 7l -) and g(x') over the interval \a,b] (Figure 9.4.2); the greater the area, 
the greater the overall error. 



Figure 9.4.2 

The area between the graphs of/and g over [a,b] measures the error in approximating/by g over [a, b]. 


Although 1 is natural and appealing geometrically, most mathematicians and scientists generally favor the following 
alternative measure of error, called the mean square error. 


f b 

mean square error = j [/ (x) 


- gfcO ] 2 dx 


Mean square error emphasizes the effect of larger errors because of the squaring and has the added advantage that it allows 
us to bring to bear the theory of inner product spaces. To see how, suppose that/is a continuous function on \ a , that we 
want to approximate by a function g from a subspace VTof C\a, &"|, and suppose that C\a, £■] is given the inner product 

{f ,g)= I f(x)g(x)dx 
Ja 

It follows that 

f-b 

2 / 2 
||f-g|| ={f-g, f-g}= / [/ (x) - g(x) ] dx = mean square error 

Ja 

so minimizing the mean square error is the same as minimizing ||f — g|| 2 . Thus the approximation problem posed informally 
at the beginning of this section can be restated more precisely as follows: 


Least Squares Approximation 

Least Squares Approximation Pro Let/be a function that is continuous on an interval \a, b ], let C\a, b] have the 
inner product 


{f, g)=/ f{x)g{x)dx 


and let IT be a finite-dimensional subspace of C[a, & ]. Find a function g in W that minimizes 

l|f-g|| 2 = A/0O-g(*)] 2 ^ 

Ja 





9 

Since ||f — g|| and llf-sll are minimized by the same function g, the preceding problem is equivalent to looking for a 
function g in W that is closest to/. But we know from Theorem 6.4.1 that g = proj^f is such a function (Figure 9.4.3). Thus 
we have the following result. 





f = fimciicm in C\u. 
to be approximated 


pmj. L T - lerasi squares 


iipproxtmuUiLHi 

W s vector apace of to f J rom W 


approximating 

functions 


Figure 9.4.3 


Solution of the Least Squares Approximation Problem If/is a continuous function on [a, b], and IT is a 

finite-dimensional subspace of C[a, b], then the function g in W that minimizes the mean square error 

[/ CO - gfcO ] 2 dx 

is g = projjr^f, where the orthogonal projection is relative to the inner product 

{f ,g)= I f(x)g(x)dx 
Ja 

The function g = proj^f is called the least squares approximation to /from IT. 

Fourier Series 



A function of the form 

r(x) = eg -I- cj cos x A- C2 cos 2x A -1- c u cos nx 

4- d i sin x A L^2sin2j:H- \- d^svanx 

is called a trigonometric polynomial, if c n and d n are not both zero, then r(x) is said to have order n. For example, 

r(x) = 2 -I- cos x — 3 cos 2x A- 7 sin4x 

is a trigonometric polynomial with 

CQ = 2, Cl = l,C2 = — 3 , £73 = 0 ,( 74 = 0 , [^1 = 0 , d'l = 0 , ( 3?3 = 0 , ( 3?4 = 7 

The order of r (^) is 4. 


( 2 ) 


It is evident from 2 that the trigonometric polynomials of order n or less are the various possible linear combinations of 

1, cosx, cos 2x,..., cosrtx, sinx, sin 2x,..., sin«x ^ 

It can be shown that these 2« -F 1 functions are linearly independent and that consequently, for any interval [a,b], they form 
a basis for a (2 n A- 1)-dimensional subspace of C[a, &]. 


Let us now consider the problem of finding the least squares approximation of a continuous function j ( ? -) over the interval 
[0, 2 tt] by a trigonometric polynomial of order n or less. As noted above, the least squares approximation to/from IT is the 
orthogonal projection of/on IT. To find this orthogonal projection, we must find an orthonormal basis gn, gi, ..., gjn f° r IT, 
after which we can compute the orthogonal projection on IT from the formula 


projfpf = (f, g 0 )go + (f, gi)gi + -+ {f, S2h)S 2n 


( 4 ) 




[see Theorem 6.3.5]. An orthonormal basis for W can be obtained by applying the Gram-Schmidt process to the basis 3, 
using the inner product 

/■2 7T 

{f>g}=/ f(x)g(x)dx 
Jo 

This yields (Exercise 6) the orthonormal basis 

1 


50 = 


fi^’ fi 


?1 = —COS X, = —1= COS MX, 


f 


g«+l = -j=sinx,..., g 2 H = -^sin«x 

w 77 v 


If we introduce the notation 


2 f 1 ill 

^0 rr— I f ? SOl? ^1 t — f f ? §1 1? ---? f ? Sh 

II U 7i I I u 7 T I | 


1 


^1 /—j^> Sm+1 Ji 

v"l 


1 


by! — j— [f> §2H 

S~\ I 


then on substituting 5 in 4, we obtain 


( 5 ) 


( 6 ) 


= ^4 [a i cos x 4“ "■ 4 ctyi cos fix ] I [^i sin x 4- I by^ sm fix ] 


(7) 


where 


In short, 


a 0 


= ~k=\ i ’ Soi = -7= f f (x)~j= dx = ^ j f (x)dx 

|/2jt 1 f fix Jo fix h Jo 


f,sl J“ fil 

/ (x)—j= cos x dx = — / 

) * Jo 

In 

f (x)cos x dx 

fi g ") = 7?J 

;■ 2“ 

f / (*) —— cos dx = — 

0 fi J 

r 2n 

j /(x)cQSftxdx 
f 0 

f „ l_ 1 

* ? Sh-|- 1 1 j— 

\ Y* 

r 2?r 

7 / / (V) —==■ sm x dx = — 

■Jo fi ^ 

/ f(x)smxdx 

h 

f £2 h = -7= 

1 V K -I 

2 7T 

/ / (*)—shim = — j 

/o ,l i 

r 2tt 

' /(x)smftxdx 

0 




/■ 2tt f 2 t 

— — I f (x) cos kx dx, bk = — j /(x)smkxdx 

77 Jo % Jo 


( 8 ) 


The numbers a$,a i, ...,a n ,b\i • • •, b n are called the Fourier coefficients of/. 


Least Squares Approximations 


Find the least squares approximation of f (x) = x ° n [0, 2 tt] by 




a trigonometric polynomial of order 2 or less; 


2 a trigonometric polynomial of order n or less. 


Solution (a) 




= lj 2 "f(x)dx = U 2 \ 

11 Jo ' l Jo 


xdx = 2 k 


For k = 1, 2,integration by parts yields (verify) 

■ 2 - 


bk 


-n: 

-u 


f (x) cos kx dx 


f (x) sin kx dx 


-if 

“ Jo 

-if 

11 Jo 


x cos kx dx = 0 


x sin kx dx = — — 
k 


Thus the least squares approximation to x on [0, 2x] by a trigonometric polynomial of order 2 or less is 

or, from 9a, 9b, and 9c, 


x ~ H a i cos x 4- a 2 cos 2x f- b i sin x 4- sin 2x 


x c± 7T — 2 sin x — sin 2x 


Solution (b) 


The least squares approximation to x on [0, 2x] by a trigonometric polynomial of order n or less is 

or, from (9a), (9b), and(9c), 


x ~ T [flicosxH-hfl H cosMx] + [&i sinx -I- ■■■ -I- b n sin^x] 


*~^-2[sm* + -^ + -^ I.+ “ 


2 3 ' ' « 

The graphs of y = x and some of these approximations are shown in Figure 9.4.4. 



Figure 9.4.4 


(9a) 


(9b) 

(9c) 


It is natural to expect that the mean square error will diminish as the number of terms in the least squares approximation 











/ (x) — -1- E (L3.it cos kx 4 bk sinArx) 

-■■ 2. k=l ■ 

increases. It can be proved that for functions/in C[0, 2~] , the mean square error approaches zero as n - * 4 - 00 ; this is 

denoted by writing 

OO 

/ (x) — -df 4- £ (t3^ cos kx + bfc sinArx) 

' 1 fc=l 

The right side of this equation is called the Fourier series for/over the interval [ 0, 2k] ■ Such series are of major importance 
in engineering, science, and mathematics. 

♦ 



Jean Baptiste Joseph Fourier (1768-1830) was a French mathematician and physicist who discovered the Fourier series 
and related ideas while working on problems of heat diffusion. This discovery was one of the most influential in the 
history of mathematics; it is the cornerstone of many fields of mathematical research and a basic tool in many branches of 
engineering. Fourier, a political activist during the French revolution, spent time in jail for his defense of many victims 
during the Terror. He later became a favorite of Napoleon and was named a baron. 


Exercise Set 9.4 




Click here for Just Ask! 


Find the least squares approximation of j (x) = 1 4 . x over the interval [ 0, 2k] by 


(a) a trigonometric polynomial of order 2 or less 


(b) a trigonometric polynomial of order n or less 


Find the least squares approximation of / (x) = x over the interval [0, 2k] by 


2. 





(a) a trigonometric polynomial of order 3 or less 


(b) a trigonometric polynomial of order n or less 


3. 

(a) Find the least squares approximation of x over the interval [0, 1] by a function of the form a 4 be 7 '. 


(b) Find the mean square error of the approximation. 


4. 

(a) Find the least squares approximation of g x over the interval [0, 1] by a polynomial of the form ag q a\x- 


(b) Find the mean square error of the approximation. 


5. 

(a) Find the least squares approximation of s in ~x over the interval [-1, 1] by a polynomial of the form 

I a\x f (32* 2- 

(b) Find the mean square error of the approximation. 


Use the Gram-Schmidt process to obtain the orthonormal basis 5 from the basis 3. 

6 . 


Carry out the integrations in 9a, 9b, and 9c. 

7. 


Find the Fourier series of f {x)= it —x over the interval [0, 2k] ■ 

8 . 


Find the Fourier series of f (*) = 1, 0 < x < it and f (x) =Q,x < x <2 tt over the interval [0, 2~]. 

9. 


What is the Fourier series of sin(3x)? 

10 . 
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9 In this section we shall study functions in which the terms are squares of 

variables or products of two variables. Such functions arise in a variety of 
QUADRATIC FORMS applications, including geometry, vibrations of mechanical systems, 

statistics, and electrical engineering. 


Quadratic Forms 

Up to now, we have been interested primarily in linear equations—that is, in equations of the form 

a\x\ 4-132*2 H-f — & 

The expression on the left side of this equation, 

fll*l + 02*2 + 

is a function of n variables, called a linear form. In a linear form, all variables occur to the first power, and there are no 
products of variables in the expression. Here, we will be concerned with quadratic forms, which are functions of the form 

2 2 2 

a\x^ -l-fl2*2 4-f a n x n 4 (all possible terms of the for i < f) ^ 

For example, the most general quadratic form in the variables jq and 7;2 is 

a\x\ + a 2 x\ + fl 3*l*2 (2) 

and the most general quadratic form in the variables 7 : 1 , 7 : 2 , and 7:3 is 

2 2 2 

fl l*l + «2*2 + fl 3*3 + ^4* 1*2 + 5*1*3 + a 6 x 2*3 (3) 

The terms in a quadratic form that involve products of different variables are called the cross-product terms. Thus, in 2 the 
last term is a cross-product term, and in 3 the last three terms are cross-product terms. 


If we follow the convention of omitting brackets on the resulting 1x1 matrices, then 2 can be written in matrix form as 


[*1 * 2 ] 


a 1 133/2 


*1 


and 3 can be written as 


[*1 *2 *3] 


J <33/2 

a 2 

[* 2 _ 



34/2 

a $ / 2 

■*r 

134/2 

a 2 

i 3 g / 2 

*2 

1 

[-0 

ctfii 2 

"3 

*3 


(4) 


(5) 


(verify by multiplying out). Note that the products in 4 and 5 are both of the form x T j\x, where x is the column vector of 
variables, and A is a symmetric matrix whose diagonal entries are the coefficients of the squared terms and whose entries off 
the main diagonal are half the coefficients of the cross-product terms. More precisely, the diagonal entry in row i and column 
i is the coefficient of x 2 , and the off-diagonal entry in row i and column j is half the coefficient of the product XjXy Here are 
some examples. 


Matrix Representation of Quadratic Forms 
















2x 2 + 6xy-7y 2 = [x y] 


4x 2 — 5y 2 — [* 7] 


xy=[ x y] 


4 0 

0 -5 


x 2 -Vlxj — 'ix 2 -\-4x\X2-2x\x^-\-Sxyx 2 = [ x \ x 2 *3] 


1 

2 

- f 

■*r 

2 

7 

3 

x 2 

- 1 

3 

-3 

*3 


♦ 


Symmetric matrices are useful, but not essential, for representing quadratic forms. For example, the quadratic form 
2x 2 4- 6xy — ly 2 , which we represented in Example 1 as x^Ax with a symmetric matrix A, can also be written as 

2x 2 4- 6xy - ly 2 = [* y] 

where the coefficient 6 of the cross-product term has been split as 5 + 1 rather than 3 -I- 3, as in the symmetric representation. 

However, symmetric matrices are usually more convenient to work with, so when we write a quadratic form as x ^Ax* it will 

always be understood, even if it is not stated explicitly, that A is symmetric. When convenient, we can use Formula 7 of 

Section 4.1 to express a quadratic form x^Ax in terms of the Euclidean inner product as 

T T 

x Ax = Ax ■ x or by symmetry of the dot product, x Ax = x - Ax 

If preferred, we can use the notation u - v = {u, v) for the dot product and write these expressions as 


2 5 

"x" 

1 -7 

y 


x^Ax = x^fAx) = {Ax, x) = (x. Ax) 


( 6 ) 


Problems Involving Quadratic Forms 

The study of quadratic forms is an extensive topic that we can only touch on in this section. The following are some of the 
important mathematical problems that involve quadratic forms. 


Find the maximum and minimum values of the quadratic form s P if x is constrained so that 

11*11 = (*1 +*2 +-+ *n) 1/2 = 1 

What conditions must A satisfy in order for a quadratic form to satisfy the inequality x ^Ax > 0 for a ll x * 0? 


If x ^Ax is a quadratic form in two or three variables and c is a constant, what does the graph of the equation x ^Ax = c 
look like? 


If P is an orthogonal matrix, the change of variables x = Py converts the quadratic form x ^Ax to 

(Py) T A(P y) —y T (P T AP) y. But P T AP is a symmetric matrix if A is (verify), so v T (P T AP)y is a new quadratic form in 

the variables ofy. It is important to know whether P can be chosen such that this new quadratic form has no 
cross-product terms. 






















In this section we shall study the first two problems, and in the following sections we shall study the last two. The following 
theorem provides a solution to the first problem. The proof is deferred to the end of this section. 


THEOREM 9.5.1 


Let A be a symmetric n x n matrix with eigenvalues Aj > A 2 > ■■■ > A„- !f x is constrained so that ||i|| — 1, then 

(a) X l >x T Ax>X yi - 

(b) x^Ax > A h if x is an eigenvector of A corresponding to \ n and x — Aj tf x is an eigenvector of A 

corresponding to Aj. 


follows from this theorem that subject to the constraint 

IMI = (*l +*2 H -b^) 1/2 = 1 

the quadratic form x T Ax has a maximum value of Aj (the largest eigenvalue) and a minimum value of A„ (the smallest 
eigenvalue). 


EXAMPLE 2 Consequences of Theorem 9.5.1 

Find the maximum and minimum values of the quadratic form 

*1 +*2 + 4 * 1*2 

subject to the constraint ^2 , and determine values of x\ and x j at which the maximum and minimum occur. 


Solution 


The quadratic form can be written as 

x'f + x ~2 + 4x1*2 — x^Alx = [*1 


^ 2 ] 


'1 2' 

'*f 

_2 1_ 

_*2_ 


The characteristic equation of A is 


det(A/ — j 4) = det 


A- 1 

-2 


-2 

A- 1 


= A 2 — 2A — 3 = (A — 3) (A 4 1) = 0 


Thus the eigenvalues of A are A = 3 and A = — 1, which are the maximum and minimum values, respectively, of the 
quadratic form subject to the constraint. To find values of x\ and *2 at which these extreme values occur, we must find 
eigenvectors corresponding to these eigenvalues and then normalize these eigenvectors to satisfy the condition ^ _ ). 


We leave it for the reader to show that bases for the eigenspaces are 


A = 3: 



A = - 1: 


1 

- 1 


Normalizing these eigenvectors yields 


















Thus, subject to the constraint x ^ + x ^ = \, maximum value of the quadratic form is A = 3 , which occurs if X \ = 1 / \[2> 

X2 = \ i an d the minimum value is A = — 1 , which occurs if ^ = ] / ^2, X2 = — 1 / \j2- Moreover, alternative bases for 
the eigenspaces can be obtained by multiplying the basis vectors above by -1. Thus the maximum value, A = 3, also occurs if 
xi = - 1 / |/2» x 2 = -l/|/2; similarly, the minimum value, A= - T also occurs if^ = - 1 / fe, X2 = 1 / ^2- 

♦ 


DEFINITION 


A quadratic form x ^Ax is called positive definite if x > 0 for all x 0 , and a symmetric matrix A is called a positive 
definite matrix if x ^Ax is a positive definite quadratic form. 

The following theorem is an important result about positive definite matrices. 


THEOREM 9.5.2 


A symmetric matrix A is positive definite if and only if all the eigenvalues of A are positive. 


)Of Assume that A is positive definite, and let A be any eigenvalue of A. If x is an eigenvector of A corresponding to A, 
then x * 0 and Ax = Ax, so 


0 < x T Ax. = x \\x = Ax r x - A||x || 2 


(7) 


where ||x|| is the Euclidean norm of x. Since ||x || 2 >0 it follows that A>0, which is what we wanted to 
show. 

Conversely, assume that all eigenvalues of A are positive. We must show that x r Ax > 0 f° r all x * Q. But if x & 0, we can 
normalize x to obtain the vector y = x / ||x|| with the property ||y|| = 1. It now follows from Theorem 9.5.1 that 

y^Av > A„ > 0 


where A„ is the smallest eigenvalue of A. Thus, 


Y = 



Multiplying through by ||x || 2 yields 


x 7 ’Ax>0 


IMI 2 


which is what we wanted to show. 


Showing That a Matrix Is Positive Definite 

















In Example 1 of Section 7.3, we showed that the symmetric matrix 


A = 


4 2 
2 4 
2 2 


2 

2 

4 


has eigenvalues A = 2 and A = 8 - Since these are positive, the matrix A is positive definite, and for all x * 0, 

Ax = 4 *i + 4*2 -I- 4x| 4- 4 x 1*2 4 4* 1*3 4- 4 * 2*3 > 0 


♦ 


Our next objective is to give a criterion that can be used to determine whether a symmetric matrix is positive definite without 
finding its eigenvalues. To do this, it will be helpful to introduce some terminology. If 



< 311 

<312 ■ 


<321 

322 ' 

" a 2n 

< 3 h 1 

< 3 h 2 ■ 



is a square matrix, then the principal submatrices of A are the submatrices formed from the first r rows and r columns of A 
for * = 1, 2,n. These submatrices are 




412 = 


'an 

<321 

312" 
*3 22 

^3 = 





a n 

321 

a 3l 


3 12 ai3 
<322 <323 
a 32 a 33 


AH 

312 ■ 


321 

322 ■ 

" a 2n 

fl H l 

3h2 l 



THEOREM 9.5.3 


A symmetric matrix A is positive definite if and only if the determinant of every principal submatrix is positive. 


We omit the proof. 


Working with Principal Submatrices 


The matrix 



-3 

4 

9 


is positive definite since 


| 2 | = 2 , 





-3 

4 

9 


= 1 


all of which are positive. Thus we are guaranteed that all eigenvalues of A are positive 


and x J Ax > 0 for all x * Q. 


Remark A symmetric matrix A and the quadratic form x are called 


positive semidefinite if x T Ax > 0 for all * 





















negative definite 


if x 7 Ax < 0 for x * 0 


negative semidefinite if x 7 As < 0 for all x 

indefinite if x 7 Ax has both positive and negative values 

Theorems Theorem 9.5.2 and Theorem 9.5.3 can be modified in an obvious way to apply to matrices of the first three types. 
For example, a symmetric matrix A is positive semidefinite if and only if all of its eigenvalues are nonnegative. Also, A is 
positive semidefinite if and only if all its principal submatrices have nonnegative determinants. 

Optional 


Proof of Theorem 9.5.1a Since A is symmetric, it follows from Theorem 7.3.1 that there is an orthonormal basis for gf 
consisting of eigenvectors of A. Suppose that S= {vj, \' 2 ,..., v H } i s suc h a basis, where v,- is the eigenvector corresponding 
to the eigenvalue Aj- If {, ) denotes the Euclidean inner product, then it follows from Theorem 6.3.1 that for any x in R n , 

x = {x, Vi }vi F {x, v 2 )v 2 4- - 4 {X, v„)v„ 

Thus 

Ax = (x, vi )A\’i 4 (x, v 2 }Av 2 4 - 4 (x, v H }Av„ 

= (x, vijAivi 4 (x, v 2 JA 2 v 2 4-F (x, v H }A H v H 

- Ai(x, vi }vi 4 A 2 (x. v 2 Jv 2 4 - 4 A„(x. v„}v„ 

It follows that the coordinate vectors for x and Ax relative to the basis S are 

00tf = ({*. >T}. (x,v 2 (x, v„}) 

( j4x )s'= ( A l{*> v l}> X 2{*’ v 2}> M X ’ v n}) 

Thus, from Theorem 6.3.2c and the fact that ||x|| = 1, we obtain 

ll x lP — ( x > V 1 + ( x > v 2} 2 3-b (x, v H } 2 = 1 

jx, AkJ = Ai(x, vi J 2 4 A 2 {x, v 2 J 2 4 - F A„(x, v H J 2 

Using these two equations and Formula 6, we can prove that x r Ax<Ai as follows: 

X 7 Ax = jx, a) = A 1 (x,v 1 } 2 + A 2 ( x, v 2 } 2 4-F A„{ x, v H } 2 

< Aj^x, vi J 2 4 Aj{x, v 2} 2 4-1- Aj{x, v H J 2 

= A i ({x, vi } 2 4 (x, v 2 } 2 44 (x, v h} 2 ) 

= Ai 

The proof that A„<x r At is similar and is left as an exercise. 


Proof of Theorem 9.5.1b If x is an eigenvector of A corresponding to Aj and ||x|| = 1, then 

x^Ax = jx, AtJ = jx, AjxJ = Aijx, xj = Aj||x|| 2 = Aj 
Similarly, x r At = A„ if INI = 1 and x is an eigenvector of A corresponding to a h - 



Exercise Set 9.5 




Click here for Just Ask! 


Which of the following are quadratic forms? 


(a) r? — ^2xy 


(b) 5xj — 2^2 + 4*1*2 

4xj — 3x|-f x| — 5xjX3 
^ x 2 — lx] + x| 4- 4x iX 2%2 


(e) x 1 x 3 — 3x 1 x 3 4- 2 x 3 x 3 
® x^ — 6*2x\ — 5x2 

(g) (x\-3x2) 2 

( h ) (xi-X 3) 2 + 2(xi+4x-j ) 2 

Express the following quadratic forms in the matrix notation x T Ax, where A is a symmetric matrix. 

2 . 


(a) 3x?4 7x? 

( b) 4x^-9xj- 6 xix 2 

(c) 5x^4 5 xiX2 


(d) -7 xjX2 



Express the following quadratic forms in the matrix notation x where A is a symmetric matrix. 


3. 


9 x^—x^-l 4*9 )-6x1x2 — 8x1x3 H-X2X3 
(b) I-x^ — 3x| — 5 xiX 2 I 9 x 1 x 3 


(c) X1X2 + X1X3 I X2X3 


^ ^ 2 xj — ^ 3 x 3 + 2^2 xjX2 — 8^3xiX3 

Xj I X2 — x| — x^ 4 2xiX2 — 10xiX4 I 4x3x4 


In each part, find a formula for the quadratic form that does not use matrices. 

4. 


(a) 


[* y] 


2 -3 

3 5 


(b) 

[*1 * 2 ] 


7 

5 

2 


5 

2 

0 


*1 

*2 


(C) 

'1 

0 

O' 

\t " 

[x y z l 

0 

-3 

0 



0 

0 

5 

z 


(d) 


[*1 x 2 *3] 



1 

2 

6 

3 


*1 

x 2 

*3 


'0 

1 

1 

f 

~ x i~ 

1 

0 

1 

1 

*2 

1 

1 

0 

1 

x 3 

1 

1 

1 

0 

x 4 


[xi x 2 x 3 x 4 ] 



In each part, find the maximum and minimum values of the quadratic form subject to the constraint x j | x ? = anc * 
determine the values of xj and X 2 at which the maximum and minimum occur. 


^ 7 x? 4 4^9 + *1*2 
5xf 4 2*2 — xi ^2 
^ 2xj 4 x| 4 3 xiX2 

In each part, find the maximum and minimum values of the quadratic form subject to the constraint | x } f | x ^=\, and 

6 . i z j 

determine the values of xi, X 2 , and X 3 at which the maximum and minimum occur. 


Xj-I X2 I 2x2~2xiX2 4 4 xiX 3 44x2x3 
^ 2xj d X 2 fx| 4 2 x 1 x 3 H 2xjX2 
3xj 4 2x^4 3x| I 2 x 1 x 3 

Use Theorem 9.5.2 to determine which of the following matrices are positive definite. 

7. 


(a) 


2 3 

3 2 


(b) 


5 - 1 
-1 5 


(c) 


2 -2 

-2 - 1 


Use Theorem 9.5.3 to determine which of the matrices in Exercise 7 are positive definite. 

8 . 


9. 


Use Theorem 9.5.2 to determine which of the following matrices are positive definite. 



(a) 


3-1 0 

-1 2-1 
0-1 3 


(b) 


0 1 1 
1 0 1 
1 1 0 


(c) 


1 2 1 
2 1 1 
1 1 3 


Use Theorem 9.5.3 to determine which of the matrices in Exercise 9 are positive definite. 

10 . 

In each part, classify the quadratic form as positive definite, positive semidefinite, negative definite, negative 
11 * semidefinite, or indefinite. 


<b) 

(c) O1-X2) 2 

(d) -( Xl -x 2 ) 2 

(e) x 2 x 2 
*i *7 

(f) * 1*2 

In each part, classify the matrix as positive definite, positive semidefinite, negative definite, negative semidefinite, or 
12. indefinite. 


(a) 


3 

0 

0 


0 0 

-2 0 

0 1 




13. 


(b) 


-5 0 0 
0 0 0 
0 0 1 


(c) 


6 7 1 

7 9 2 
1 2 1 


(d) 


-4 7 8 

7-3 9 

8 9-1 


(e) 


0 0 0 
0 0 0 
0 0 0 


(f) 


1 0 0 
0 1 0 
0 0 1 


Let x be a quadratic form in x i, X 2 , • ••, and define 7: R n —* R by 7(x) — x T Ax. 


(a) Show that 7(x 4- v) = 7(x) H 2x 7 Ay h Tty). 

(b) Show that 7(kx) = k 2 7(x). 

(c) Is T a linear transformation? Explain. 

In each part, find all values of k for which the quadratic form is positive definite. 

x \ f £*2 “4*1*2 

(b) 5x 2 kkx^ h 4 x\X2 — 2x\X2 — 2x\X2 
^ 3xf 4 x 2 4- 2x\ f 2x\X2 4 2^2*3 


14 . 






Express the quadratic form (c^j | c jZ'a +- — I c H x„) 2 in the matrix notation where A is symmetric. 


Let x = (x!, x 2 ,x„)- hi statistics, the quantity 

x = £(*l + X2 + -+x„) 
is called the sample mean of x\, X 2 , ■■■, x n , and 

* 1 = n ]_ ! [(*1 -*) 2 + (* 2-*) 2 + -+ (* H -*) 2 ] 
is called the sample variance. 


(a) Express the quadratic form in the matrix notation x where A is symmetric. 

(b) Is a positive definite quadratic form? Explain. 


Complete the proof of Theorem 9.5.1 by showing that a < x ^Ax if || x || = 1 and a = x ^Ax if x i s an eigenvector of A 

-i n M — ■ Yl - 

1 '• corresponding to A„- 


Discussion 

Discovery 


Indicate whether each statement is true (T) or false (F). Justify your answer. 

18. 


(a) A symmetric matrix with positive entries is positive definite. 

(b) x 2 _ x 2 + x 2 + 4 ^^ is a quadratic form. 

(c) (Aq — 3 x 2 ) 2 is a quadratic form. 

(d) A positive definite matrix is invertible. 

(e) A symmetric matrix is positive definite, negative definite, or indefinite. 

(f) If A is positive definite, then — A is negative definite. 


Indicate whether each statement is true (T) or false (F). Justify your answer. 

19. 


(a) If jc is a vector in R 71 , then xx is a quadratic form. 





(b) If x is a positive definite quadratic form, then so is x ^ -1 v . 


(c) If A is a matrix with positive eigenvalues, then xTAx is a positive definite quadratic 
form. 

(d) If A is a symmetric 2x2 matrix with positive entries and a positive determinant, then A 
is positive definite. 

(e) If x^Ax is a quadratic form with no cross-product terms, then A is a diagonal matrix. 

(f) If x^Ax is a positive definite quadratic form in x and y, and if c ^ 0, then the graph of 
the equation x ^Ax = c is an ellipse. 


What property must a symmetric 2x2 matrix A have for x r Ax = 1 to represent a circle? 

20 . 


Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 



9.6 

DIAGONALIZING 
QUADRATIC FORMS; 
CONIC SECTIONS 


In this section we shall show how to remove the cross-product terms from a 
quadratic form by changing variables, and we shall use our results to study 
the graphs of the conic sections. 


Diagonalization of Quadratic Forms 


Let 


■v A=[^l X2 - x„] 


3 11 312 
321 322 


3 1m 
32m 


3m 1 3 m 2 3 mh 


*1 

*2 


( 1 ) 


be a quadratic form, where A is a symmetric matrix. We know from Theorem 7.3.1 that there is an orthogonal matrix P that 
diagonalizes A; that is, 

"Ai 0 - 0 


P 1 AP = D = 


where Ap A 2 > • ••, A H are the eigenvalues of A. If we let 


V = 


0 A 2 
0 0 


Al 

yi 


y n 


and make the substitution x = Py in 1, then we obtain 


x J Ax — (Py ) J APy = y T P T APy = y 1 Dy 


But 



'Ai 

0 ■ 

•• 0 " 

>r 

y T Dy=[y\ A2 - Am] 

0 

A 2 ■ 

•• 0 

A2 


0 

0 ■ 

■■ A h 

Ah 


— 3—i 

which is a quadratic form with no cross-product terms. 


In summary, we have the following result. 


THEOREM 9.6.1 


Let x 7^^ [y e a quadratic form in the variables xp X 2 > ■■■, where A is symmetric. IfP orthogonally diagonalizes A, and if 
the new variables yp y-,, ... y M are defined by the equation x = py, then substituting this equation in x T Ax yields 
















x j4x — y Dy — + -^ 2 X 2 —I - AvlVv] 

where Ai, A 2 , ■■■, A„ are the eigenvalues of A and 

"Ai 0 - o' 


d=p t ap= 

0 a 2 - 0 



0 0 - A„ 






The matrix P in this theorem is said to orthogonally diagonalize the quadratic form or reduce the quadratic form to a sum of 
squares. 


Reducing a Quadratic Form to a Sum of Squares 

Find a change of variables that will reduce the quadratic form ^2 _ ^2 _ ^ + 4 ^^ to a sum of squares, and express the 

quadratic form in terms of the new variables. 


Solution 

The quadratic form can be written as 



1 

-2 

O' 

■*r 

1— 

to 

LO 

-2 

0 

2 

*2 


0 

2 

- 1 

*3 


The characteristic equation of the 3 x 3 matrix is 

A-1 2 


0 


2 A -2 

0 —2 A d 1 


= A — 9A = A(A + 3) (A — 3) = 0 


so the eigenvalues are A = 0> A = — 3, A = 3- We leave it for the reader to show that orthonormal bases for the three 
eigenspaces are 


A = 0: 


"2 " 


1 " 


2 " 

3 


3 


3 

1 

3 

(Ai 

1 

II 

2 

3 

A = 3: 

2 

3 

2 


2 


1 

3 


3 


3 


Thus, a substitution x = Py that eliminates cross-product terms is 


"*f 


2 1 _ 2 

3 3 3 

'Af 

*2 

= 

1 2 2 

3 3 3 

A2 

*3 


2 2 1 

A3 



3 3 3 



or, equivalently, 


*1 = §ai - = 

*2 = 5 AI - f A2 + fA3 

*3 - §A1 +- §A2 + jA3 


The new quadratic form is 

























[71 72 73] 


or, equivalently, 


0 

0 

0 

71 

0 

-3 

0 

72 

0 

0 

3 

73 


— 3,^2 + 3^1 


♦ 


Remar There are other methods for eliminating the cross-product terms from a quadratic form; we shall not discuss them 
here. Two such methods, Lagrange's reduction and Kronecker's reduction, are discussed in more advanced books. 

Conic Sections 


We shall now apply our work on quadratic forms to the study of equations of the form 

0,7?" 4- 2 bxy I cy 2 -I- dx 4- ey -I / = 0 

where a, b, .. .,/are real numbers, and at least one of the numbers a, b, c is not zero. An equation of this type is called a 
quadratic equation in x and y, and 

ax 2 4 2 bxy 4 cy 2 


is called the associated quadratic form. 


( 2 ) 


EXAMPLE 2 Coefficients in a Quadratic Equation 


In the quadratic equation 

3x 2 4- 5 xy - 1y 2 4- 2x 4 7 = 0 

the constants in 2 are 


a = 3, b = c = — 1, 

2 

d = 2, e = 0, f=l 

♦ 

EXAMPLE 3 Examples of Associated Quadratic Forms 


Quadratic Equation 

Associated Quadratic Form 



3x 2 4- 5xy - ly 2 1 2x 1-7 = 0 

3x 2 + 5xy — ly 2 



4x 2 - 5y 2 f 8y [■ 9 = 0 

4 x 2 -5y 2 



*7 + 7 = 0 

*7 



♦ 


Graphs of quadratic equations in x and y are called conics or conic sections. The most important conics are ellipses, circles, 
hyperbolas, and parabolas; these are called the nondegenerate conics. The remaining conics are called degenerate and include 













single points and pairs of lines (see Exercise 15). 


A nondegenerate conic is said to be in standard position relative to the coordinate axes if its equation can be expressed in one 
of the forms given in Figure 9.6.1. 


(«) 
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Parabola 
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a ' = k y, k-*k> 

Parabola 






Figure 9.6.1 

































EXAMPLE 4 Three Conics 


From Figure 9.6.1, the equation 



matches the form of an ellipse with k = 2 and 1=3. Thus the ellipse is in standard position, intersecting the x-axis at (-2, 0) 
and (2, 0) and intersecting the y-axis at (0, -3) and (0, 3). 


The equation x 2 — 8y 2 = — 16 can be rewritten as y 2 S 2 — x 2 / 16 = 1 , which is of the form y 2 / £ 2 — x 2 / P" — 1 with £ = ^3, 

1 = 4 . Its graph is thus a hyperbola in standard position intersecting the y-axis at (0, — \[2) and (0, \[2)- 

The equation 5x 4 2y — 0 can be rewritten as x = — -^y, which is of the form x = ky with k——^. Since £ < 0 , its graph 
is a parabola in standard position opening downward. 

♦ 


Significance of the Cross-Product Term 


Observe that no conic in standard position has an xy-term (that is, a cross-product term) in its equation; the presence of an xy 
-term in the equation of a nondegenerate conic indicates that the conic is rotated out of standard position (Figure 9.6.2a).Also, 
no conic in standard position has both an 7 y and an x term or both a y 2 and a y term. If there is no cross-product term, the 

occurrence of either of these pairs in the equation of a nondegenerate conic indicates that the conic is translated out of standard 
position (Figure 9.6.2 b). The occurrence of either of these pairs and a cross-product term usually indicates that the conic is both 
rotated and translated out of standard position (Figure 9.6.2c). 





Figure 9.6.2 


One technique for identifying the graph of a nondegenerate conic that is not in standard position consists of rotating and 
translating the xy-coordinate axes to obtain an x'y'-coordinate system relative to which the conic is in standard position. Once 
this is done, the equation of the conic in the x y'-system will have one of the forms given in Figure 9.6.1 and can then easily be 
identified. 


EXAIV Completing the Square and Translating 

Since the quadratic equation 

2x 2 4y 2 — 12x —4y 4 18 = 0 

contains A - 2 -, x-, y 2 -, and y-terms but no cross-product term, its graph is a conic that is translated out of standard position but 

not rotated. This conic can be brought into standard position by suitably translating coordinate axes. To do this, first collect 
x-terms and y-terms. This yields 












By completing the squares* on the two expressions in parentheses, we obtain 

2(x 2 -6x4 9)4(y 2 -4y4 4)=: -13+18 1-4 
or 

2(x — 3) 2 + (y — 2) 2 —4 


If we translate the coordinate axes by means of the translation equations 

x' = x — 2, y ! =y-2 

then 3 becomes 

n -) f2 / 2 

2/ 2 + y 2 = 4 or V+4r- 


= 1 


which is the equation of an ellipse in standard position in the xV'-system. This ellipse is sketched in Figure 9.6.3. 



Figure 9.6.3 


j2 


2 



= 1 


(3) 


Eliminating the Cross-Product Term 


We shall now show how to identify conics that are rotated out of standard position. If we omit the brackets on 1 x 1 matrices, 
then 2 can be written in the matrix form 


a b 

'x' 


"x" 

b c 

7 

4 [d e] 

y 


4/ = 0 


or 


where 


x T Ax + / = 0 



K=[d e] 


Now consider a conic C whose equation in xy-coordinates is 

x T Ax hKx + f = 0 

We would like to rotate the xy-coordinate axes so that the equation of the conic in the new x^y '-coordinate system has no 
cross-product term. This can be done as follows. 


Step 1. Find a matrix 




















that orthogonally diagonalizes the matrix A. 


Step 2. Interchange the columns of P, if necessary, to make dct(P) — 1- This ensures that the orthogonal coordinate 
transformation 


is a rotation. 


x = Px*, that is 



Step 3. To obtain the equation for C in the *y-system, substitute 5 into 4. This yields 


(5) 


(Px f ) T A(Px f ) + K(Px f ) + / = 0 

or 


(x') T (P T AP)x ' + (KP)x’ + f= 0 


Since P orthogonally diagonalizes A, 


p t ap= 


Ai 

0 


0 

^2 


where Ai and A 2 are eigenvalues of A. Thus 6 can be rewritten as 


"Ai 

0 " 

V 


>11 PU 

V 

0 

A 2 

/_ 

4- [ d e ] 

P2\ P22 

y_ 


or 

\\x' 2 4 A zy f2 4 d'x' 4 e > y t 4 / = 0 

(where d f = dpu + ep2i and e* = dpn + eP 22 )- This equation has no cross-product term. 


The following theorem summarizes this discussion. 


( 6 ) 


THEOREM 9.6.2 


Principal Axes Theorem for R 2 

Let 

ax' 2 4 2bxy 4 cy A 4 dx 4 ey 4-./ = 0 

be the equation of a conic C, and let 

xTaIx — ax 2 4 2 bxy + cy 2 

be the associated quadratic form. Then the coordinate axes can be rotated so that the equation 
for C in the new x'y-coordinate system has the form 

Ai*' 2 +A 9 y 2 +^y+ e y+./=o 

where Ai and A 2 are the eigenvalues of A. The rotation can be accomplished by the substitution 

X = Px’ 



















where P orthogonally diagonalizes A and det(P) = 1- 


Eliminating the Cross-Product Term 


9 "7 

Describe the conic C whose equation is 5x — 4xy A 8 y —36 = 0. 


Solution 

The matrix form of this equation is 


x T Ax-36 = 0 


(7) 


where 


A = 


5 -2 

-2 8 


The characteristic equation of A is 


det(A/ — j 4) = det 


A— 5 2 

2 A— 8 


= (A — 9)(A —4) = 0 


so the eigenvalues of A are A = 4 and A = 9- We leave it for the reader to show that orthonormal bases for the eigenspaces are 


A = 4: \'i = 


2/{5 
1 / {5 


A = 9: V2 = 


- 1 / {5 
2/{5 


Thus 


P = 


2/fi -1//5 


1 / 2/ {5 

orthogonally diagonalizes A. Moreover, det(i- J ) = 1, and thus the orthogonal coordinate transformation 


( 8 ) 


is a rotation. Substituting 8 into 7 yields 

(Px f ) T A(Px f ) - 36 = 0 or 


(x f ) T (P T AP)x f - 36 = 0 


Since 


p t ap= 


4 0 
0 9 


this equation can be written as 



[4 ol 

-1 

--L 

_1 

[* y l 

OA 1 

0 

y_ 


-36 = 0 


or 

00 /2 /2 

4x ,z + 9/ - 36 = 0 or = 1 

y 4 

which is the equation of the ellipse sketched in Figure 9.6.4. In that figure, the vectors vj and V2 are the column vectors of 
P —that is, the eigenvectors of A. 
























Figure 9.6.4 


a 


9 



= 1 


♦ 


Eliminating the Cross-Product Term Plus Translation 


Describe the conic C whose equation is 

5x 2 -4xy + Sy 2 + 4{5x - + 4 = 0 


Solution 

The matrix form of this equation is 

where 

As shown in Example 6, 


x 4 Kx + 4 = 0 


A = 


5 -2 
2 8 


and K = [V 5-16/5] 


P = 


2 1 

f5 ~f5 

1 2 

f5 f5 


orthogonally diagonalizes A and has determinant 1. Substituting x — p x ! into 9 gives 

(Px f ) T A(Px / ) 4 K(Px f ) + 4 = 0 


or 


Since 


(x') T (P T AP)x r + (KP)x' + 4 = 0 


P T AP = 


’4 0" 


' 20 

30 " 

_0 9_ 

and KP = 

f 5 



2 1 

1 2 

f5 f5 


= [-8 -36] 


(9) 


( 10 ) 


10 can be written as 


















4*' 2 + 9/ 2 - 8 *' - 36 y f 4 4 = 0 


(ID 


To bring the conic into standard position, the x t y t axes must be translated. Proceeding as in Example 5, we rewrite 11 as 

4(*' 2 -2x') + 9<y 2 -4/)= -4 

Completing the squares yields 

4(* /2 — 2x* 4 1) +9(/ 2 _4/ + 4) = - 44 4 + 36 
or 

4(x^ 1) 2 4 9(y^2) 2 = 36 (12) 

If we translate the coordinate axes by means of the translation equations 

i //_ / 0 

a — a i y — v £ 

then 12 becomes 

i i 

4*" +9/' =36 or + ^^ = 1 

which is the equation of the ellipse sketched in Figure 9.6.5. In that figure, the vectors \q and are the column vectors of P 
—that is, the eigenvectors of A. 



Figure 9.6.5 




= 1 


♦ 


Exercise Set 9.6 




Click here for Just Ask! 


In each part, find a change of variables that reduces the quadratic form to a sum or difference of squares, and express the 
quadratic form in terms of the new variables. 


(a) 

2* 2 1 

1 2*2 — 2*1*2 

(b) 

5x\ 1 

1 2*2 4- 4*1*2 











(c) 2x1*2 


(d) — 3x 2 4 5x| 4- 2*1*2 

In each part, find a change of variables that reduces the quadratic form to a sum or difference of squares, and express the 

2 . quadratic form in terms of the new variables. 

3x 2 4 4*| 4 5*| I 4*i* 2 —4*2*3 
(k) 2x 2 I 5*^4 5*| 4 4 *i *2 — 4 *i *3 — 8 * 2*3 

— 5*j 4 *2 — *f 4 6 * 1 * 344 * 1*2 
(d) 2 * 1*3 +■ 6 * 2*3 

Find the quadratic forms associated with the following quadratic equations. 

3. 

(a) 2 * 2 - 3*7 | 4 ^ 2 — 7* 4 2y 4 7 = 0 

(b) * 2 - *7 4 5* 4 87 - 3 = 0 

(c) 5*7 = 8 

(d) 4x 2 -2.y 2 = 7 

(e) y 2 +. 7 * -87 -5 = 0 

Find the matrices of the quadratic forms in Exercise 3. 

4. 

Express each of the quadratic equations in Exercise 3 in the matrix form x ^ £ 4 ; \ f = 0 

Name the following conics. 

6. 

(a) 2 x 2 4 57 2 = 20 



(b) 4x 2 4 9y 2 = 1 

(c) x 2 -y 2 -S = 0 

(d) 4y 2 — 5* 2 = 20 

(e) x 1 | y 1 - 25 = 0 

(f) 7y 2 -2;r = 0 

(g) -* 2 = 2y 

(h) 3x — \ ly 2 — 0 

(i) y-x 2 = 0 

(j) x 2 -3 = -7 2 

In each part, a translation will put the conic in standard position. Name the conic and give its equation in the translated 
7. coordinate system. 

(a) 9x 2 I 4y 2 — 36* — 24y I 36 = 0 

(b) x 2 - \6y 2 4 Sx f 128y = 256 

(c) y 2 -Sx-\4y 4-49 = 0 

(d) x 2 I y 2 4-§x — HDy 4 18 = 0 

(e) 2* 2 -3.y 2 4 6x f 20 y= -41 

(f) x 2 h 10* I ly= -32 


8. 


The following nondegenerate conics are rotated out of standard position. In each part, rotate the coordinate axes to remove 
the xy-term. Name the conic and give its equation in the rotated coordinate system. 



(a) 2x 2 -4xy-y 2 I 8 = 0 


(b) 5x 2 4 4xy f- 5y 2 = 9 


(c) Ux 2 + 24xy4- 4y 2 - 15 = 0 

In Exercises 9-14 translate and rotate the coordinate axes, if necessary, to put the conic in standard position. Name the conic 
and give its equation in the final coordinate system. 

9x 2 - 4xy 4- 6y 2 — lOx — 20.y = 5 

9 • 


3x 2 - 3xy -12 y 2 - 30x - 64,y = 0 

10 . 


11 . 


2x 2 - 4 xy-y 2 -4x-3y= - 14 


21x 2 \-6xy I I3y 2 -U4x I 34,y 4-73 = 0 
x 2 - 6xy - ly 2 4 10x4 2y I 9 = 0 


4x 2 -20xy I 25y 2 - 15x-6y = 0 


The graph of a quadratic equation in x and y can, in certain cases, be a point, a line, or a pair of lines. These are called 
15. degenerate conics. It is also possible that the equation is not satisfied by any real values of x and y. In such cases the 
equation has no graph; it is said to represent an imaginary conic. Each of the following represents a degenerate or 
imaginary conic. Where possible, sketch the graph. 


(a) *2_^2 = 0 

(b) x 2 3^ 2 f 7 = 0 

(c) 8x 2 4 ly 2 = 0 

(d) x 2 -2 xy f y 2 = 0 



(e) 9x 2 A- UxyA Ay 2 - 52 = 0 

(f) x 2 \ y 2 -2x-Ay = - 5 


16 . 


Prove: If £> ^ 0, then the cross-product term can be eliminated from the quadratic form ax ~ 4 . 2bxy A- cy 2 by rotating the 
coordinate axes through an angle 0 that satisfies the equation 


cot 20 = 


a — c 
2b 
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q 7 In this section we shall apply the diagonalization techniques developed in 

' the preceding section to quadratic equations in three variables, and we 

QUADRIC SURFACES shall use our results to study quadric surfaces. 


In Section 9.6 we looked at quadratic equations in two variables. 


Quadric Surfaces 

An equation of the form 

ax 2 4- by 2 -I- cz 2 4 2 dxy 4 2 exz 4 2 fyz H gx 4 ky H iz 4./ = 0 

where a, b ,.. .,/are not all zero, is called a quadratic equation in x, y, and z; the expression 

ax 2 + &y 2 4 cz 2 4 2<sfx,y 4 2exz 4 2/.yz 

is called the associated quadratic form, which now involves three variables: x, y, and z. 


Equation 1 can be written in the matrix form 


[* y A 

a d 

d b 

e 

/ 

-1 

S; * 

_i 

4 [g h i] 

1 

1_ 


e f 

c 

z 


z 


or 


where 


x t Ax 4 Kx 4 j = 0 



~x " 


1 

B 

_i 

X = 

z 

, A = 

^ <3“ 

1 _ 


( 1 ) 


Associated Quadratic Form 


The quadratic form associated with the quadratic equation 

3x 2 4 2y 2 — z 2 4 Axy 4 3 xz — 3yz 4 lx 4 2y 4 3z — 7 = 0 
is 

3x 2 4 2.y 2 -z 2 4 Axy V 3xz - 8yz 

♦ 

Graphs of quadratic equations in x, y, and z are called quadrics or quadric surfaces. The simplest equations for quadric 
surfaces occur when those surfaces are placed in certain standard positions relative to the coordinate axes. Figure 9.7.1 
shows the six basic quadric surfaces and the equations for those surfaces when the surfaces are in the standard positions 
shown in the figure. If a quadric surface is cut by a plane, then the curve of intersection is called the trace of the plane on the 
surface. To help visualize the quadric surfaces in Figure 9.7.1, we have shown and described the traces made by planes 
parallel to the coordinate planes. The presence of one or more of the cross-product terms xy, xz , and yz in the equation of a 

















quadric indicates that the quadric is rotated out of standard position; the presence of both x 2 and x terms, y l and y terms, or 
z 2 and z terms in a quadric with no cross-product term indicates the quadric is translated out of standard position. 


Surface 


EtfUitSioin 


Sin face 


Equation 


1: 111 [iiOi d 


T -J $ 

jf y k z ! 

TT + ~5 + ~ = 1 


Elliptic cone 


Jt y- 
1 i7J 


T3ie traces mi ilic coordinate 
plane* are ellipse-s, as ara lEte 
lnn.es ill those planes that are 
parallel to the cwrdinale planer 
and intmcct the surface in more 
l 3 ii.ui one point. 




The I race in ihe vv-plarie is a 
point (ille origin). and die inild 
in planes parallel Ki the vy-plane 
,ire ellipses. '1 lie iraces m ihc .v: 
and Ar-pJimes are pair* of lines 
ijilerieeLing at die origin. The 
traces in planes pam Ik-3 tu die^e 
are hvpeffeobs 


Hyperboloid 
of one sheet 



Elliptic paraboloid 




The mice in the u-plane rsnn 
ellipse, asnire the traces in 
planes parallel to the sy-plane. 
Tin- traces in the v.--plane and 
die a:- plane are hyperbolas, as 
are die traces in those plane* rh :le 
arc parallel Co these and do not 
puss through the ory- 
intercepts, At these mTereepis 
che traces ate pairs of 
intersect ini! lines. 



The trace in llie u-plane is a 
point (the origin), and Che traces 
in planes parallel to and above 
llie ry-pikmc arc eHip.ses.The 
traces in I he v,-- and i planes 
21 rc pa mho I as, ;i.s are the traces 
in planes parallel to these 


v 


Hyperboloid 
of two sheets 



Hyperbolic paraboloid 



* 

x~ 




y 


There ss no trace in the jrv-plane 
In planes parallel to llie iy-plane 
chat intersect the surface in more 
than one point, lhc trace!; are 
ellipses.. In the y-- and xz- planes. 
tEiiC traeCH are hyperbola*. a* arc 
the traces in (hose planes ih^t are 
parallel to chew and intersect die 
surface in mote than one prune 



rhe trace in the vv-pStinc is a 
pair of linen intersecting lhc 

origin. The traces in plants 
pu m I k I to i he .yv-p lane are 
hyperbolas. The hyperbolas 
above llie .yv-plane open in the 
y-tii ration, and chose he low m 
llie r-direehon. I he UiWCS in lhe 
>and .t:-plane* are parabola*, 
as are the traces in planes 
parallel to these. 


Figure 9.7.1 


Identifying a Quadric Surface 


Describe the quadric surface whose equation is 


Ax 


i 


2 


.2 




























Solution 


Rearranging terms gives 
Completing the squares yields 


4(x 2 -4x) 4- 360 2 -6y) -9 z 2 = -304 


or 


or 


4(x 2 -4x4-4)4-36(y 2 -6y h 9) - 9z 2 = -304 P 16 + 324 


4(> - 2) 2 + 360 ~ 3) 2 - 9z 2 = 36 


0-2)- , z- 


9 


+ 0-3) -V = l 


Translating the axes by means of the translation equations 


x f — x ~ 2, y'=y-3. 


z =z 


yields 


/ 2 ,r 2 

—=1 

9 y 4 


which is the equation of a hyperboloid of one sheet. 


Eliminating Cross-Product Terms 

The procedure for identifying quadrics that are rotated out of standard position is similar to the procedure for conics. Let Q 
be a quadric surface whose equation in ^^-coordinates is 


x 1 I Kx 4 j = 0 


( 2 ) 


We want to rotate the xyz-coordinate axes so that the equation of the quadric in the new x’y’z* -coordinate system has no 
cross-product terms. This can be done as follows: 


Step 1. Find a matrix P that orthogonally diagonalizes x ^4x- 


Step 2. Interchange two columns of P, if necessary, to make det (p) — This ensures that the orthogonal coordinate 
transformation 


is a rotation. 


x = Px f , that is. 



(3) 


Step 3. Substitute 3 into 2. This will produce an equation for the quadric in x f y V-coordinates with no cross-product 
terms. (The proof is similar to that for conics and is left as an exercise.) 


The following theorem summarizes this discussion. 








THEOREM 9.7.1 


Principal Axes Theorem for ff 3 

Let 

ax 2 4 - by 2 -I- cz 2 4- 2dxy 4 2exz h 2 fyz 4-gx A hy \-iz I- j — 0 

be the equation of a quadric Q,and let 

x^j4x — ax 2 -I- by 2 -I- cz 2 4- 2is?xy 4 2exz f 2/yz 

be the associated quadratic form. The coordinate axes can be rotated so that the equation of Q 
in the xVz'-coordinate system has the form 

X\x >2 4 A zy f 2 4 A 3 z* 2 4- gV 4- h > y > 4- i V 4 j — 0 

where Ai, X2, and A3 are the eigenvalues of A. The rotation can be accomplished by the 
substitution 

X = Pxf 

where P orthogonally diagonalizes A and det(/>) = 1. 


Eliminating Cross-Product Terms 


Describe the quadric surface whose equation is 

4x 2 4 4 y 2 4 4z 2 4 4 xy 4 4xz I- 4 yz -3 = 0 


Solution 


The matrix form of the above quadratic equation is 


x r J 4x-3 = 0 


(4) 


where 


A = 


4 2 2 
2 4 2 
2 2 4 


As shown in Example 1 of Section 7.3, the eigenvalues of A are A = 2 and A = 8> and A is orthogonally diagonalized by the 
matrix 


P = 


- 1//2 -\iy[6 \!y[2 

1 / y[2 - 1 / 1 / /3 

0 21^2 1/^3 


where the first two column vectors in Z 3 are eigenvectors corresponding to \ — 2, and the third column vector is an 
eigenvector corresponding to A = 8- 


Since det(P) = 1 (verify), the orthogonal coordinate transformation x = p x ' is a rotation. Substituting this expression in 4 
yields 









or, equivalently. 


(Px 1 ) T A(Px ') -3 = 0 


fx') T (P T AP)x' -3 = 0 
But 

so 5 becomes 

[*' 

or 

This can be rewritten as 

which is the equation of an ellipsoid. 

Exercise Set 9.7 

Click here for Just Ask! 

Find the quadratic forms associated with the following quadratic equations. 

1 . 

(a) x 2 -|- 2y 2 — z 2 + Axy — 5yz 4- 7x 4- 2z = 3 

(b) 3x 2 -|- 7z 2 4- 2x.y - 3xz -I- - 3x = 4 

(c) + xz 4- 7^ = 1 

(d) x 2 H-7 2 -z 2 = 7 

(e) 3z 2 +3xz-147 +9 = 0 



P T AP = 


2 0 0 
0 2 0 
0 0 8 



'2 

0 

0 " 

1- 

*T- 

1_ 

/ A 

0 

2 

0 

/ 


1 

o 

0 

-1 

CO 

Z' f 


-3 = 0 


2x' 2 +27 ,2 H 8z' 2 = 3 


3/2 3/2 3/8 


(5) 


♦ 


(f) 2z 2 4 2xz 4- 7 2 I 2x — y + 3z = 0 















Find the matrices of the quadratic forms in Exercise 1. 

2. 

Express each of the quadratic equations given in Exercise 1 in the matrix form K -^bc | Kx 4 j = 0- 

3. 

Name the following quadrics. 

4. 

(a) 36x 2 4 9y 2 -\ 4z 2 -36 = 0 

(b) 2x 2 q 6y 2 -3z 2 = \S 

(c) 6x 2 — 3y 2 — 2z 2 — 6 = 0 

(d) 9x 2 -\ 4y 2 —z 2 = 0 

(e) \6x 2 + y 2 = \6z 

(f) lx 2 — 3y 2 hz = 0 

(g) x 2 b.y 2 \ z 2 = 25 

In Exercise 4, identify the trace in the plane z = 1 in each case. 

5. 

Find the matrices of the quadratic forms in Exercise 4. Express each of the quadratic equations in the matrix form 
6 * s r A4li + j = 0- 

In each part, determine the translation equations that will put the quadric in standard position, and find the equation of th 
7. quadric in the translated coordinate system. Name the quadric. 

(a) 9x 2 | 36y 2 -\ 4z 2 -\Sx-\44y-24z \ 153 = 0 

(b) 6x 2 I 3y 2 — 2z 2 I 12x - 18y - 8z= -7 

(c) 3x 2 - 3 y 2 -z 2 4 42x f 144 = 0 


(d) 4x 2 4 9y 2 -z 2 -54y-50z=544 



(e) x 2 f 16y 2 f 2x- 32y - 16z- 15 = 0 


(f) 7x 2 — 3y 2 h 126x I 72,y + z4- 135 = 0 

(g) x 2 -I y 1 4- z 2 - 2x V 4y — 6z = 11 


In each part, find a rotation * = p x ! that removes the cross-product terms, and give its equation in the x ! y' V-system. 
Name the quadric. 


(a) 2x 2 H 37 2 v 23z 1 \- 72 xz I 150 = 0 


(b) 4x 2 -\ 4y 2 4- 4z 2 I 4 xy I 4 xz I 47z-5 = 0 

(c) 144^ 2 + 1007 2 f 81z 2 -216xz-540;r-720z=0 


(d) 2^:7 -I z — 0 

In Exercises 7-10 translate and rotate the coordinate axes to put the quadric in standard position. Name the quadric and give 
its equation in the final coordinate system. 

2x7 -|- 2 xz | 27Z — 6x — 67 — 4z = — 9 

9. 


lx 2 H 77 2 f 10z 2 — 2x7 — 4xz I 47Z — 12x E 127 H- 60z = 24 

10 . 


2x7-6x + IO7 | z — 31 = 0 

11 . 


12 . 


2x 2 4- 2y 2 4- 5z 2 — 4x7 — 2xz I 27 Z 4- lOx — 267 — 2z = 0 


Prove Theorem 9.7.1. 

13. 
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9.8 


In this section we shall discuss some practical aspects of solving systems of 
linear equations, inverting matrices, and finding eigenvalues. Although we have 
previously discussed methods for performing these computations, we now 
consider their suitability for the computer solution of the large-scale problems 
that arise in real-world applications. 


COMPARISON OF 
PROCEDURES FOR 
SOLVING LINEAR 
SYSTEMS 


Counting Operations 

Since computers are limited in the number of decimal places they can carry, they round off or truncate most numerical quantities. 

o 

For example, a computer designed to store eight decimal places might record y as either .66666667 (rounded off) or .66666666 
(truncated).* In either case, an error is introduced that we shall call roundoff error or rounding error. 

The main practical considerations in solving linear algebra problems on digital computers are minimizing the computer time (and 
thus cost) needed to obtain the solution, and minimizing inaccuracies due to roundoff errors. Thus, a good computer algorithm 
uses as few operations and memory accesses as possible, and performs the operations in a way that minimizes the effect of 
roundoff errors. 

In this text we have studied four methods for solving a linear system, of n equations in n unknowns: 

Gaussian elimination with back-substitution 


2. Gauss-Jordan elimination 


Computing A~^ then forming x = A~^b 


Cramer's rule 


To determine how these methods compare as computational tools, we need to know how many arithmetic operations each requires. 
It is usual to group divisions and multiplications together and to group additions and subtractions together. Divisions and 
multiplications are considerably slower than additions and subtractions, in general. We shall refer to either multiplications or 
divisions as “multiplications” and to additions or subtractions as “additions.” 

In Table 1 we list the number of operations required to solve a linear system = b of n equations in n unknowns by each of the 
four methods discussed in the text, as well as the number of operations required to invert A or to compute its determinant by row 
reduction. 


Table 1 


Operation Counts for an Invertible n x n Matrix A 


Method 


Number of Additions Number of Multiplications 


Solve = b by Gauss- Jordan elimination 


3 2 6 







Method 

Number of Additions 

Number of Multiplications 

Solve j4x = b by Gaussian elimination 

i„ 3 +_ |„ 

jK 3 + « 2 — 

Find ^-1 by reducing [A \ I] to [/ | A~ { ] 

ft 3 — 2ft 2 4- ft 

n' 3 

Solve Ax = h as x = 21 

a 

LO 

1 

as 

to 

ft 3 

1 n 2 

Find det(A) by row reduction 

¥ - i* 2+ b 

l* 3 4- 


Solve j4x = b by Cramer's rule 

b 4 -h 3 -i' ,2+ b 


2 2,2 
jK -4 - 1 


Note that the text methods of Gauss-Jordan elimination and Gaussian elimination have the same operation counts. It is not hard to 
see why this is so. Both methods begin by reducing the augmented matrix to row-echelon form. This is called the forward phase or 
forward pass. Then the solution is completed by back-substitution in Gaussian elimination and by continued reduction to reduced 
row-echelon form in Gauss-Jordan elimination. This is called the backward phase or backward pass. It turns out that the number 
of operations required for the backward phase is the same whether one uses back-substitution or continued reduction to reduced 
row-echelon form. Thus the text method of Gaussian elimination and the text method of Gauss-Jordan elimination have the same 
operation counts. 


Remark There is a common variation of Gauss-Jordan elimination that is less efficient than the one presented in this text. In our 
method the augmented matrix is first reduced to reduced row-echelon form by introducing zeros below the leading l's; then the 
reduction is completed by introducing zeros above the leading l's. An alternative procedure is to introduce zeros above and below 
a leading 1 as soon as it is obtained. This method requires 


y-f addition and y + y 
both of which are larger than our values for all n > 3. 


multiplications 


To illustrate how the results in Table 1 are computed, we shall derive the operation counts for Gauss-Jordan elimination. For this 
discussion we need the following formulas for the sum of the first n positive integers and the sum of the squares of the first n 
positive integers: 


1 + 2 + 3 h -h« 


n{n + 1) 
2 


( 1 ) 


l 2 + 2 2 + 3 2 + - + * 2 


n{n + 1) (2 n + 1) 
6 


( 2 ) 


Derivations of these formulas are discussed in the exercises. We also need formulas for the sum of the first n — \ positive integers 
and the sum of the squares of the first n — 1 positive integers. These can be obtained by substituting n — \ for n in I and 2. 


1 + 2 + 3-I-b (yi — 1) — 


{n — 1)« 
2 


(3) 


l 2 + 2 2 + 3 2 + - + (*-l) 2 = 


(n — 1 )«( 2 « — 1 ) 
6 


(4) 


Operation Count for Gauss-Jordan Elimination 

Let j4 x = b be a system of n linear equations in n unknowns, and assume that A is invertible, so that the system has a unique 
solution. Also assume, for simplicity, that no row interchanges are required to put the augmented matrix |b] in reduced 














row-echelon form. This assumption is justified by the fact that row interchanges are performed as bookkeeping operations on a 
computer (that is, they are simulated, not actually performed) and so require much less time than arithmetic operations. 

Since no row interchanges are required, the first step in the Gauss-Jordan elimination process is to introduce a leading 1 in the first 
row by multiplying the elements in that row by the reciprocal of the leftmost entry in the row. We shall represent this step 
schematically as follows: 
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Note that the leading 1 is simply recorded and requires no computation; only the remaining n entries in the first row must be 
computed. 

The following is a schematic description of the steps and the number of operations required to reduce [j 4 | b] to row-echelon form. 
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Thus, the number of operations required to complete successive steps is as follows: 
Steps 1 and la. 

2 

Multiplications: — 1) = w 

2 

Additions: n{n—\)=n —n 



Steps 2 and 2a. 


Steps 3 and 3a. 


Multiplications: (h — 1) + (n — 1) (« — 2) = (w — l) 2 

Additions: («— 1)(« — 2) = (« — l) 2 — (« — 1) 


Multiplications: (» — 2) + (« — 2) (« — 3) = (« — 2) 2 

Additions: (a — 2) (m — 3) = (n — 2) 2 — (« — 2) 


Steps _ l) and (n — \)a. 


Step n. 


Multiplic ations: 4 ( = 2 2 ) 

Additions: 2( = 2 2 —2) 


Multiplications: 1 ( = l 2 ) 

Additions: 0(=1 2 — 1) 


Therefore, the total number of operations required to reduce [j 4 | b] to row-echelon form is 
Multiplications: n 2 + (w — l) 2 4- (« — 2) 2 H-h l 2 

Additions: [n 2 f (» - 1) 2 + (h- 2) 2 + - + l 2 ] -[» + (»-!) + (h - 2) +-+ 1] 


or, on applying Formulas 1 and 2, 


Multiplications: "("+D0i»+1) = 

6 3 2 6 


Additions: 


«(« 4- l)(2w 4- 1) _ n{n 4- 1) « 

6 2 _ 3 3 


(5) 

( 6 ) 


This completes the operation count for the forward phase. For the backward phase we must put the row-echelon form of [j4 | b] 
into reduced row-echelon form by introducing zeros above the leading l's. The operations are as follows: 
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Thus, the number of operations required for the backward phase is 

Multiplications: (« — 1) + (m — 2) + ■ 
Additions: (« — 1) + (m — 2) 4- ■ 

or, on applying Formula 3, 


Multiplications: 


(« — !)« 


■+ 2 + 1 
■H 2 + 1 


n_ 

2 


(V) 


Addto: (»-D» = ai_a 


(8) 


Thus, from 5, 6, 7, and 8, the total operation count for Gauss-Jordan elimination is 



















Multiplications: 


(9) 


Additions: 



( 10 ) 


Comparison of Methods for Solving Linear Systems 

In practical applications it is common to encounter linear systems with thousands of equations in thousands of unknowns. Thus we 
shall be interested in Table 1 for large values of n. It is a fact about polynomials that for large values of the variable, a polynomial 
can be approximated well by its term of highest degree; that is, if * 0, then 

+ -1- ~ for large x 

(Exercise 12). Thus, for large values of n, the operation counts in Table 1 can be approximated as shown in Table 2. 


Table 2 

Approximate Operation Counts for an Invertible nxn Matrix A for Large n 


Method 

Number of Additions 

Number of Multiplications 

Solve j4\ = b by Gauss-Jordan elimination 

n 3 

« 3 

~ 3 

~ 3 

Solve j4x = 1) by Gaussian elimination 


^* 3 


~ 3 

~ 3 

Find j 4 -1 by reducing [A\I] to [/|^ _1 ] 


KJ« 3 

Solve Ax = b as x _ a _1 b 

R,* 3 

^ 3 

Find det(2l) by row reduction 

^ 3 

^ 3 


~ 3 

~ 3 

Solve j4x = b by Cramer's rule 




~ 3 

~ 3 


It follows from Table 2 that for large n, the best of these methods for solving ^ = b are Gaussian elimination and Gauss-Jordan 
elimination. The method of multiplying by ^-1 is much worse than these (it requires three times as many operations), and the 
poorest of the four methods is Cramer's rule. 


Remark 

We observed in the remark following Table 1 that if Gauss-Jordan elimination is performed by introducing zeros above and 
below leading l's as soon as they are obtained, then the operation count is 

3 2 

pr- A pr- multiplications 

which is 50% greater than the / 3 multiplications required by 

It is reasonable to ask if it is possible to devise other methods for solving linear systems that might require significantly fewer than 
the ~ yp / 3 additions and multiplications needed in Gaussian elimination and Gauss-Jordan elimination. The answer is a qualified 
“yes.” In recent years, methods have been devised that require rj Crfi multiplications, where q is slightly larger than 2.3. However, 
these methods have little practical value because the programming is complicated, the constant C is very large, and the number of 






—— — additions and 


Thus, for large n, this procedure requires 
the text method. Similarly for additions. 


tP / 2 multiplications, 













additions required is excessive. In short, there is currently no practical method for the direct solution of general linear systems that 
significantly improves on the operation counts for Gaussian elimination and the text method of Gauss-Jordan elimination. 

Operation counts are not the only criterion by which to judge a method for the computer solution of a linear system. As the speed 
of computers has increased, the time it takes to move entries of the matrix from memory to the processing unit has become 
increasingly important. For very large matrices, the time for memory accesses greatly exceeds the time required to do the actual 
computations! Despite this, the conclusion above still stands: Except for extremely large matrices, Gaussian elimination or a 
variant thereof is nearly always the method of choice for solving Ax = b- It is almost never necessary to compute A~K and we 
should avoid doing so whenever possible. Solving ^ = 1, by Cramer's rule would be senseless for numerical purposes, despite its 
theoretical value. 


Avoiding the Inverse 

Suppose we needed to compute the product A5 -1 Cx- The result is a vector y. Rather than computing y = A{B~^ (Cx)) as given, it 
would be more efficient to write this as 

z = B~ l Cx 
y = Az 

that is, as 

Bz = Cx 
y = Az 

and to compute the result as follows: First, compute the vector w = Cx ; second, solve Bz = w for z using Gaussian elimination; 
third, compute the vector v = Al. 

♦ 

For extremely large matrices, such as the ones that occur in numerical weather prediction, approximate methods for solving j4 x = b 
are often employed. In such cases, the matrix is typically sparse; that is, it has very few nonzero entries. These techniques are 
beyond the scope of this text. 


Exercise Set 9.8 



Click here for Just Ask! 


Find the number of additions and multiplications required to compute AB if A is an m x n matrix and B is an m x p matrix. 

1 . 

Use the result in Exercise 1 to find the number of additions and multiplications required to compute A k by direct multiplication 
2* if A is an n x n matrix. 

Assuming A to be an n x n matrix, use the formulas in Table 1 to determine the number of operations required for the 
3- procedures in Table 3. 


n = 5 « = 10 « = 100 « = 1000 


Table 3 








+ X + X 


Solve j4x = b by Gauss- Jordan 
elimination 


Solve j4x = b by Gaussian 
elimination 


Find j 4 -1 by reducing [A\l] to 




Solve A\ = b as x = A~ l b 


Find det(j4) by row reduction 


Solve j4x = b by Cramer's rule 


Assuming for simplicity a computer execution time of 2.0 microseconds for multiplications and 0.5 microsecond for additions, 
4- use the results in Exercise 3 to fill in the execution times in seconds for the procedures in Table 4. 


Table 4 


« = 5 


« = 10 


w = 100 


« = 1000 


Solve j4x = b by Gauss- Jordan 
elimination 

Solve j4x = b by Gaussian 
elimination 


Execution Time Execution Time Execution Time Execution Time 

(sec) (sec) (sec) (sec) 


Find a~^ by reducing [A | /] to 

Solve j4x =b as x = j4 -1 b 
Find det(j4) by row reduction 


Solve j4x = b by Cramer's rule 


Derive the formula 

5. 


1 + 2 + 3 + + n — 


n{n f 1) 
2 


Hint Let s = \ 4 . 2 + 3 + ... 4 - n . Write the terms of in reverse order and add the two expressions for 





















Use the result in Exercise 5 to show that 


6. 


1+2 + 3 H- \ («-!) = 


(« — 1 )k 
2 


Derive the formula 

i 2 + 2 2 + 3 2 + ... + „ 2 = n(a±mn±ll 

using the following steps. 

(a) Show that (k + l) 3 — k 3 = 3k 2 f 3£ f 1. 

(b) Show that 

[2 3 — l 3 ] + [3 3 - 2 3 ] f [4 3 — 3 3 ] +■■■+ [(« + l) 3 -* 3 ] ={n f 1) 3 -1 

(c) Apply (a) to each term on the left side of (b) to show that 

(w+ l) 3 -l = 3fl 2 + 2 2 + 3 2 + - + * 2 l +3Tl + 2 + 3 + - + »l +» 

(d) Solve the equation in (c) for 1 2 | 2 2 + 3 2 H_| « 2 , use the result of Exercise 5, and then simplify. 


Use the result in Exercise 7 to show that 
*' l 2 2 2 + 3 2 I.I („-l) 2 = ("-1 M 2 "" 1 ) 


9. 


Let /? be a row-echelon form of an invertible n x « matrix. Show that solving the linear system fa. = h by back-substitution 
requires 


^2 ~ *2 multiplications 


and Sr-*- 


additions 


Show that to reduce an invertible rrxn matrix to / by the text method requires 

3 3 2 

^ multiplications and I ^ additions 

Afote Assume that no row interchanges are required. 


11 . 


Consider the variation of Gauss-Jordan elimination in which zeros are introduced above and below a leading 1 as soon as it is 
obtained, and let A be an invertible n x n matrix. Show that to solve a linear system Ax = b using this version of Gauss-Jordan 
elimination requires 


d multiplications and 4r — 77 


additions 


Note Assume that no row interchanges are required. 


12. 


(For Readers Who Have Studied Calculus) Show that if p (*) = fl g + a jx I_I a k x k ^ where then 


X 


lim 


+ 00 


zW = i 

k 

a k x 


This result justifies the approximation | ... _) a k x k fy a k x k for large values of v. 



13 . 

(a) Why is y = ({AB -1 ) C) x an even less efficient way to find y in Example 1 ? 


(b) Use the result of Exercise 1 to find the operation count for this approach and for v — A(B 1 (Cx)). 
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9.9 

I //-DECOMPOSITIONS 


With Gaussian elimination and Gauss-Jordan elimination, a linear system is 
solved by operating systematically on an augmented matrix. In this section 
we shall discuss a different organization of this approach, one based on 
factoring the coefficient matrix into a product of lower and upper triangular 
matrices. This method is well suited for computers and is the basis for many 
practical computer programs. * 


Solving Linear Systems by Factoring 

We shall proceed in two stages. First, we shall show how a linear system ^x — can be solved very easily once the 
coefficient matrix A is factored into a product of lower and upper triangular matrices. Second, we shall show how to 
construct such factorizations. 

If an n x n matrix A can be factored into a product of n x n matrices as 

A = LU 

where L is lower triangular and U is upper triangular, then the linear system = b can be solved as follows: 


Step 1. Rewrite the system ^x = b as 


LUx = b 


( 1 ) 


Step 2. Define a new nxl matrix y by 


Ux 


V 


( 2 ) 


Step 3. Use 2 to rewrite 1 as Ly = b and solve this system for y . 

Step 4. Substitute y in 2 and solve for x. 

Although this procedure replaces the problem of solving the single system j4x = b by the problem of solving the two systems 
Ly = b and Ux = y, the latter systems are easy to solve because the coefficient matrices are triangular. The following 
example illustrates this procedure. 


Solving a System by Factorization 


Later in this section we will derive the factorization 
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'-O 

C-J 

1_ 


‘2 0 O' 
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-3 -3 0 
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-3 10 

0 1 3 

4 9 2 


4-3 7 

0 0 1 


Use this result and the method described above to solve the system 











2 6 2 

"*f 


2 

-3 -8 0 

*2 
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2 

4 9 2 

*3 
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Solution 

Rewrite 3 as 


'2 0 O' 

'1 3 f 

"*f 


'2' 

-3 10 

0 1 3 

*2 

= 

2 

4-3 7 

-1 

O 

O 

*3 


3 


As specified in Step 2 above, define y\, y % and v?, by the equation 


'1 

3 

r 

■*r 


"71" 

0 

1 

3 

*2 

= 

72 

0 

0 

1 

*3 


73 


so 4 can be rewritten as 


"2 0 O' 


"71' 


'2' 

-3 10 


72 

= 

2 

4-3 7 


73 


3 


or, equivalently, 

2yi =2 

-371+72 =2 

4yi - 3,y 2 + 7^3 = 3 


( 4 ) 


( 5 ) 


The procedure for solving this system is similar to back-substitution except that the equations are solved from the top down 
instead of from the bottom up. This procedure, which is called forward-substitution, yields 

y 1 = 1 . 72 = 5 , 73 = 2 

(verify). Substituting these values in 5 yields the linear system 


'i 3 r 

■*l" 


'f 

0 1 3 

*2 

= 

5 

1 

o 

o 

1-L 

1_ 

*3 


2 


or, equivalently, 

*1 + 3^2 + *3 = 1 
*2 + 3*3 = 5 

*3 = 2 

Solving this system by back-substitution yields the solution 

*1 = 2, *2 = - 1, *3 = 2 


(verify). 


* 


LU-Decompositions 

Now that we have seen how a linear system of n equations in n unknowns can be solved by factoring the coefficient matrix, 
we shall turn to the problem of constructing such factorizations. To motivate the method, suppose that an n x « matrix A has 
been reduced to a row-echelon form U by Gaussian elimination—that is, by a certain sequence of elementary row operations 
By Theorem 1.5.1 each of these operations can be accomplished by multiplying on the left by an appropriate elementary 
matrix. Thus there are elementary matrices Ey ••••> E]r, suc h that 


Ek-E 2 EiA=U 


( 6 ) 


































By Theorem 1.5.2, E\, i?2’ •••> Ek are invertible, so we can multiply both sides of Equation 6 on the left successively by 

£* _1 .3r 1 .*r 1 

to obtain 

(7) 

In Exercise 15 we will help the reader to show that the matrix L defined by 

i=*rV~£*‘ (8) 

is lower triangular provided that no row interchanges are used in reducing A to U. Assuming this to be so, substituting 8 into 
7 yields 

A = L U 

which is a factorization of A into a product of a lower triangular matrix and an upper triangular matrix. 

The following theorem summarizes the above result. 


THEOREM 9.9.1 


If A is a square matrix that can be reduced to a row-echelon form U by Gaussian elimination without row interchanges, 
then A can be factored as A — L U> where L is a lower triangular matrix. 


DEFINITION 


A factorization of a square matrix A as A — L U, where L is lower triangular and U is upper triangular, is called an £ 77 
■decomposition or triangular decomposition of the matrix A. 


An LU-Decomposition 


Find an £ ^/-decomposition of 
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To obtain an £ ^/-decomposition, A = LZL we shall reduce A to a row-echelon form U using Gaussian elimination and then 
calculate L from 8. The steps are as follows: Thus 








































and, from 8, 
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is an ^[/-decomposition of A. 


Procedure for Finding LU-Decompositions 

As this example shows, most of the work in constructing an £[/-decomposition is expended in the calculation of L. However, 
all this work can be eliminated by some careful bookkeeping of the operations used to reduce A to U. Because we are 
assuming that no row interchanges are required to reduce A to U, there are only two types of operations involved: multiplying 
a row by a nonzero constant, and adding a multiple of one row to another. The first operation is used to introduce the leading 
l's and the second to introduce zeros below the leading l's. 


In Example 2, the multipliers needed to introduce the leading l's in successive rows were as follows: 

for the first row 
1 for the second row 
y for the third row 

Note that in 9, the successive diagonal entries in L were precisely the reciprocals of these multipliers: 


L = 



4 


0 0 

d) o 



(9) 


Next, observe that to introduce zeros below the leading 1 in the first row, we used the operations 

add 3 times the first row to the second 
add — 4 times the first row to the third 

and to introduce the zero below the leading 1 in the second row, we used the operation 

add 3 times the second row to the third 

Now note in 10 that in each position below the main diagonal of L, the entry is the negative of the multiplier in the operation 
that introduced the zero in that position in U : 


/, = 


2 

-3 
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0 

I 

—3 


0 

0 

7 


( 10 ) 


We state without proof that the same happens in the general case. Therefore, we have the following procedure for 
constructing an £ [/-decomposition of a square matrix A provided that A can be reduced to row-echelon form without row 
interchanges. 


























Step 1. Reduce A to a row-echelon form U by Gaussian elimination without row interchanges, keeping track of the 
multipliers used to introduce the leading l's and the multipliers used to introduce the zeros below the leading l's. 


Step 2. In each position along the main diagonal of L, place the reciprocal of the multiplier that introduced the leading 1 
in that position in U. 


Step 3. In each position below the main diagonal of L, place the negative of the multiplier used to introduce the zero in 
that position in U. 


Finding an LlZ-Decomposition 


Find an JJJ -decomposition of 
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Solution 
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multiplier = —9 
multiplier = —3 


multiplier = | 


I -i 0 

0 1 k 

0 (0) ] 
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multiplier = —8 


multiplier = I 
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We begin by reducing A to row-echelon form, keeping track of all multipliers. Constructing L from the multipliers yields the 



















£ ^/-decomposition. 



'6 0 O' 


A = LU = 

9 2 0 

o i r 


3 8 1 

2 



0 0 1 


We conclude this section by briefly discussing two fundamental questions about ££/-decompositions: 

1. Does every square matrix have an £ [/-decomposition? 

2. Can a square matrix have more than one £ ^/-decomposition? 


We already know that if a square matrix A can be reduced to row-echelon form by Gaussian elimination without row 
interchanges, then A has an ££/-decomposition. In general, if row interchanges are required to reduce matrix A to 
row-echelon form, then there is no £ ^/-decomposition of A. However, in such cases it is possible to factor A in the form of 
PL U-decomposition 

A = PLU 

where L is lower triangular, U is upper triangular, and Pisa matrix obtained by interchanging the rows of / appropriately 
(see Exercise 17). Any matrix that is equal to the identity matrix with the order of its rows changed is called a permutation 
matrix. 


In the absence of additional restrictions, £ ^/-decompositions are not unique. For example, if 



hi 

0 

0 

"l 

“12 

“13 

A = LU = 

hi 

*22 

0 

0 

1 

“23 


hi 

*32 

*33 

0 

0 

1 


and L has nonzero diagonal entries, then we can shift the diagonal entries from the left factor to the right factor by writing 


1 0 



*31 *32 

*11 *22 


*11 

0 

0 " 

"l 

“12 

“13 

0 

*22 

0 

0 

1 

“23 

0 

0 

*33 

0 

0 
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i o o 


* 2 L 

1 

0 

*11 

* 11“12 

* 11 “ 13 

*11 

0 

*22 

* 22 “ 23 

* 2 L 

*32 

1 

0 

0 

*33 

*11 

*22 





which is another triangular decomposition of A. 


Exercise Set 9.9 


Click here for Just Ask! 






















Use the method of Example 1 and the L ^/-decomposition 


3 

-6' 


3 0" 

'1 

-2" 

-2 

5_ 


_ — 2 1_ 

_0 

1_ 


to solve the system 

3 xi — 6x9 — 0 
— 2xi-|- 5x2 = 1 


Use the method of Example 1 and the £ [/-decomposition 


3 

-6 

-3' 


3 

0 

O' 

'1 

-2 

- f 

2 

0 

6 

= 

2 

4 

0 

0 

1 

2 

-4 

7 

4 


-4 

- 1 

2 

0 

0 

1 


to solve the system 

3xi — 6 x 2 — 3x3 — — 3 
2xi I 6x3 = — 22 

— 4xi + 7x2 f 4x3 = 3 

In Exercises 3-10 find an £ ^/-decomposition of the coefficient matrix; then use the method of Example 1 to solve the 
system. 



2 

8' 

■*r 


' -2' 

3. 

_ - 1 

- 1 _ 

_* 2 _ 


_ -2_ 



' -5 

- 10' 

"*r 


' - 10 ' 

4. 

6 

5_ 

*2 


19_ 



2 -2 -2' 

r*ii 


' -4' 

5. 

0-2 2 

-1 5 2 

1 - 

LO tO 

1_ 

— 

-2 

6 


6. 

' -3 

1 

12 

-2 

-6' 

2 

"*l‘ 

*2 

_ 

' -33' 
7 


0 

1 

1 

x 3 


- 1 


5 

5 

10' 

'*1' 


'o' 

8 

-7 

-9 

*2 

= 

1 

0 

4 

26 

*3 
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8. 

' - 1 

-3 

-4' 

'*1' 


' -6' 

3 

10 

- 10 

x 2 

= 

-3 


-2 

-4 

11 

*3 
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1 

0 

1 

o' 

'*l' 


5' 

2 

3 

-2 

6 

x 2 


- 1 

0 

- 1 

2 

0 

*3 


3 

0 

0 

1 

5 

x 4 
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"2 

-4 

0 

O' 
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Y 
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2 

12 

0 

*2 


0 

0 

- 1 

-4 

-5 

*3 
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0 

0 

2 

11 

*4 
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Let 

11 . 




(a) Find an £ [/-decomposition of A. 

(b) Express A in the form A = L\DU\, where L \ is lower triangular with l's along the main diagonal, U\ is upper 
triangular, and D is a diagonal matrix. 

(c) Express A in the form A — L 2 U 2 ' w h ere Li i s lower triangular with l's along the main diagonal and [/ 2 is upper 
triangular. 


12 . 

(a) Show that the matrix 

'0 f 
.1 °_ 

has no /^-decomposition. 

(b) Find a PL [/-decomposition of this matrix. 

Let 

13. U .U 

A= a b 
[c d_ 

(a) Prove: If a * 0, then A has a unique £ [/-decomposition with l's along the main diagonal of L. 

(b) Find the /.[/-decomposition described in part (a). 


Let Ax = b be a linear system of n equations in n unknowns, and assume that A is an invertible matrix that can be 

14. reduced to row-echelon form without row interchanges. How many additions and multiplications are required to solve 
the system by the method of Example 1 ? 

Note Count subtractions as additions and divisions as multiplications. 

Recall from Theorem 1.7.1b that a product of lower triangular matrices is lower triangular. Use this fact to prove that the 

15. matrix L in 8 is lower triangular. 



Use the result in Exercise 15 to prove that a product of finitely many upper triangular matrices is upper triangular. 

16 . 

Prove: If A is any nxn matrix, then A can be factored as A — PL U, where L is lower triangular, U is upper triangular, 

17. and P can be obtained by interchanging the rows of / appropriately. 

Hint Let U be a row-echelon form of A, and let all row interchanges required in the reduction of A to U be performed 
first. 


Factor 

18, [3-10' 

A= 3-11 

0 2 1 

as A = PL Lh where Pisa permutation matrix obtained from by interchanging rows appropriately, L is lower 
triangular, and U is upper triangular. 

Show that if A = PLZL then = b ma Y be solved by a two-step process similar to the process in Example 1. Use this 

19. method to solve = b> where A is the matrix in Exercise 18 and i> = 
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Chapter 9 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be MATLAB, Mathematica, 
Maple, Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear 
algebra capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The 
goal of these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the 
techniques in these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise 
sets. 


Section 9.1 

Tl. 

(a) Find a general solution of the system 

y [ = 371 + 272 + 273 

y ! 2 = y 1 M72 + 73 
73 = -271 -472- 73 

by computing appropriate eigenvalues and eigenvectors. 

(b) Find the solution that satisfies the initial conditions 71(0) = 0 > 72(0) = L 73(0) = — 3 - 


The electrical circuit in the accompanying figure, called a parallel LRC circuit, contains a resistor with resistance R ohms 
«• <n), an inductor with inductance L henries (H), and a capacitor with capacitance C farads (F). It is shown in electrical circuit 
theory that the current I in amperes (A) through the inductor and the voltage drop V in volts (V) across the capacitor satisfy 
the system of differential equations 

dJ_ = V_ 
dt L 

dV_ _ / V 

dt C RC 

where the derivatives are with respect to the time t . Find I and V as functions of t if £ = 0 .5 H> C = 0.2 F> = 2 and the 
initial values of V and / are F(0) = 1 V and /(0) = 2 A- 

C 

r— K —1 

R 

-VSA,-<1 

L 

U—nmn- 1 

Figure Ex-T2 


Section 9.3 


Tl. (Least Squares Straight Line Fit) 











Read your documentation on finding the least squares straight line fit to a set of data points, and then use your utility to find 
the line of best fit to the data in Example 1. Do not imitate the method in the example; rather, use the command provided by 
your utility. 


T2. (Least Squares Polynomial Fit) 

Read your documentation on fitting polynomials to a set of data points by least squares, and then use your utility to find the 
polynomial fit to the data in Example 3. Do not imitate the method in the example; rather, use the command provided by your 
utility. 

Section 9.7 

Tl. (Quadric Surfaces) 

Use your technology utility to perform the computations in Example 3. 

Section 9.9 

Tl. (^-decomposition) 

Technology utilities vary widely in how they handle JJJ- and .PZ, ^/-decompositions. For example, most programs perform 
row interchanges to reduce roundoff error and hence produce a pj ^/-decomposition, even when asked for an JJJ 
-decomposition. Ready our documentation, and then see what happens when you use your utility to find an JJJ 
-decomposition of the matrix A in Example 3. 
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CHAPTER 



Complex Vector Spaces 


NTRODUCTION Up to now we have considered only vector spaces for which the scalars are real numbers. However, 
for many important applications of vectors, it is desirable to allow the scalars to be complex numbers. For example, in 
problems involving systems of differential equations, complex eigenvalues are often the case of greatest interest. 

In the first three sections of this chapter we will review some of the basic properties of complex numbers, and in subsequent 
sections we will discuss vector spaces in which scalars can be complex numbers. 
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j a j In this section we shall review the definition of a complex number and discuss 

1" ■ 1 the addition, subtraction, and multiplication of such numbers. We will also 

COMPLEX NUMBERS consider matrices with complex entries and explain how addition and 

subtraction of complex numbers can be viewed as operations on vectors. 


Complex Numbers 

Since * 2 > 0 for every real number x, the equation _ _ } has no real solutions. To deal with this problem, mathematicians of 
the eighteenth century introduced the “imaginary” number, 

i — i/ — 1 

which they assumed had the property 

i 2 = (\T^) 2 = -! 

but which otherwise could be treated like an ordinary number. Expressions of the form 


where a and b are real numbers, were called “complex numbers,” and these were manipulated according to the standard rules of 
arithmetic with the added property that j 2 = _ ]. 

By the beginning of the nineteenth century it was recognized that a complex number 1 could be regarded as an alternative symbol 
for the ordered pair 

(a,b) 

of real numbers, and that operations of addition, subtraction, multiplication, and division could be defined on these ordered pairs 
so that the familiar laws of arithmetic hold and p- _ _ \. This is the approach we will follow. 


DEFINITION 


A complex number is an ordered pair of real numbers, denoted either by ( a, b) or by a + bi, where j 2 = _ ]. 


Two Notations for a Complex Number 


Some examples of complex numbers in both notations are as follows 


Ordered Pair Equivalent Notation 


(3,4) 
(- 1 , 2 ) 
( 0 , 1 ) 


S + 4 i 
-1+2 i 
0 + i 
















Ordered Pair 

Equivalent Notation 

(2, 0) 

2 + 0 i 

(4, -2) 

44 (-2 )i 


For simplicity, the last three complex numbers would usually be abbreviated as 

0 + j = j, 2 + Oi = 2, 4 4 ( — 2)i = 4 — 2i 


Geometrically, a complex number can be viewed as either a point or a vector in the xy-plane (Figure 10.1.1). 


£f + 

I 


X 

J —1- 


iii) Complex mimberas a 
point 



ih) Complex number 
vector 

Figure 10.1.1 


#■ 


Complex Numbers as Points and as Vectors 


Some complex numbers are shown as points in Figure 10.1.2a and as vectors in Figure 10.1.2 b. 



Figure 10.1.2 





















The Complex Plane 

Sometimes it is convenient to use a single letter, such as z, to denote a complex number. Thus we might write 

z = a bi 

The real number a is called the real part of z, and the real number b is called the imaginary part of z. These numbers are denoted 
by Re (z) and Im (z), respectively. Thus 

Re (4 — 3i) =4 and Ln (4 - 3z) = - 3 

When complex numbers are represented geometrically in an ;ry-coordinate system, the x-axis is called the real axis, the y-axis is 
called the imaginary axis, and the plane is called the complex plane (Figure 10.1.3). The resulting plot is called an Argand 
diagram. 



Figure 10.1.3 

Argand diagram. 


Operations on Complex Numbers 

Just as two vectors in pf- are defined to be equal if they have the same components, so we define two complex numbers to be 
equal if their real parts are equal and their imaginary parts are equal: 


DEFINITION 


Two complex numbers, a 4- bi and c \ 

di, are defined to be equal, written 


a 4- hi — r -1- di 

if a = c and b = d- 



If y, = 0, then the complex number a 4 bi reduces to a + Oz, which we write simply as a. Thus, for any real number a, 

a— a 4- Of. 

so the real numbers can be regarded as complex numbers with an imaginary part of zero. Geometrically, the real numbers 
correspond to points on the real axis. If we have a = 0> then a + bi reduces to 0 4- bi, which we usually write as bi- These complex 
numbers, which correspond to points on the imaginary axis, are called pure imaginary numbers. 

Just as vectors in p} are added by adding corresponding components, so complex numbers are added by adding their real parts 
and adding their imaginary parts: 


{a 4- bi) 4- (c 4- di) = (a 4- c) 4- (b 4 d)i 

The operations of subtraction and multiplication by a real number are also similar to the corresponding vector operations in p 


( 2 ) 

2 - 


(i a 4- bi) — (c 4 di) = (a — c) -4 (b — d)i 


(3) 











k(a + bi) = ( ka ) 4- (kb)i, k real 


(4) 


Because the operations of addition, subtraction, and multiplication of a complex number by a real number parallel the 
corresponding operations for vectors in the familiar geometric interpretations of these operations hold for complex numbers 
(see Figure 10.1.4). 



(A) The sum of two 
complex numbers 



{ h) The difference of two 
complex numbers 



ic) The product of a complex 
number- and a positive 
real number k 



(*/) The product of a complex 
number z and a negative 
real number k 


Figure 10.1.4 











It follows from 4 that (_ l)z +z = 0 (verify), so we denote ( — 1 )z as — z and call it the negative of z. 


Adding, Subtracting, and Multiplying by Real Numbers 

If Z\ = 4 - 5; and Z2 = - 1 + 6 ;, find Z1 + Z9 , Zl -z 2 , 3z v and -z 2 . 

Solution 


z! +z 2 = (4 - 50 + ( - 1 + 60 = (4 - 1) + ( - 5 + 6); = 3 + ; 
z!-z 2 = (4-50 -(- 1 + 60 = (44- 1) + ( — 5 — 6)j = 5 — Hi 
3zi = 3(4-50 = 12-15; 

-Z2 = ( - 1>2 = (-!)(- 14 60 = 1 - 6 ; 


♦ 


So far, there has been a parallel between complex numbers and vectors in However, we now define multiplication of complex 
numbers, an operation with no vector analog in R 2 . To motivate the definition, we expand the product 

(a + &0(c + £3f0 

following the usual rules of algebra but treating j 2 as -1. This yields 

(i a + hi) {c + di) = ac +- bdi * 1 4 adi + bci 
= (ac — bd) + {ad 4 bc)i 


which suggests the following definition: 


(a + bi ) (c + di) = (ac — bd) + (ad 4 bc)i 


(5) 


Multiplying Complex Numbers 


(3 4 20(4 4 50 = (3 - 4 — 2 - 5) I- (3-5 + 2-4); 

= 24 23; 

(4-0(2-30 = [4-2-(-l)(-3)] + [(4)(-3) + (-l)(2)]; 
= 5-14; 

i 2 = (0 4- 0(0 + 0 = (0 - 0 - 1 -1) + (0 - 1 + 1 - 0); = - 1 


We leave it as an exercise to verify the following rules of complex arithmetic: 

zi +Z2=Z2+zi 
z\Z2=z 2 z\ 

Zl + (z 2 + zi) = (zi 4 z 2 ) + Z 3 
Zl (z 2 z 3 ) = (ziz 2 )z 3 
Zl (z 2 + zi) = z\z 2 + z iz 3 
0 4 z = z 
z h (-z) = 0 
1 -z = z 






These rules make it possible to multiply complex numbers without using Formula 5 directly. Following the procedure used to 
motivate this formula, we can simply multiply each term of a 4 bi by each term of c -F di, set j 2 = _ l, and simplify. 


Multiplication of Complex Numbers 


(3 + 2i)(4 hi) = 12 + 3; + 8;d 2 i 2 = 12 -f lli — 2 = 10 F Ui 
5 —-i;J(2 + 3;) = 10+ 15i-i-|i 2 = 10 F 14H | = ^ + 14i 

i(l + 0(1 ~2i) = i(l -2i + i-2i 2 ) = j(3 -0 = 3i - i 2 = 1 F 3i 


Remark Unlike the real numbers, there is no size ordering for the complex numbers. Thus, the order symbols <, <, >, and > are 
not used with complex numbers. 


Now that we have defined addition, subtraction, and multiplication of complex numbers, it is possible to add, subtract, and 
multiply matrices with complex entries and to multiply a matrix by a complex number. Without going into detail, we note that the 
matrix operations and terminology discussed in Chapter 1 carry over without change to matrices with complex entries. 


Matrices with Complex Entries 


If 


then 


A = 


1 — i 

1 4 i 4 — i 


and 


B = 


2 — 3i 


1 — i 
4 


A 1 B = 

1 4- i 

1 — 2 i 

3-2i 

8 — i 



■ 2 

iA = 

i 

— i 


i 4 i 2 

4i — i^ 

AB = 

1 

— i 


1 4 i 

4-i 


A — B = 


1 — i -1 
-1-1-4 i —i 


i 1 

— 1 + i 1 + 4i 


i 1 — i 
2-3 i 4 

1 - j + ( — i) - (2 — 3i) l-(l-0 + (-0-4 

(l+i)-i+ (4-i)-(2-3i) (1 + j). (1 _i) 4 (4 -i)-4 


- 3 -i 1-5 i 

4 — 13 i 18-4 i 


Exercise Set 10.1 



























In each part, plot the point and sketch the vector that corresponds to the given complex number. 

1 . 

(a) 2 + 3 i 

(b) -4 

(c) - 3 - 2 i 

(d) -5; 

Express each complex number in Exercise 1 as an ordered pair of real numbers. 

2 . 

In each part, use the given information to find the real numbers x and y. 

3. 

(a) x — iy = —2 -\-3i 

(b) (x + 7 ) 4* (x-y)i = 3 + i 

Given that Zl = 1 - 2i and ?2 = 4 _| 5 j, find 

4. 

(a) z\ I zi 

(b) z\ ~ z 2 

(c) 4z\ 

(d) — z 2 

(e) 3z\ I 4 z 2 


5 . 


In each part, solve for z. 



(a) z + (1 — j) = 3 4- 2; 


(b) — 5z — 5 4- 10; 

(c) (i—z) + (2z~3i) = — 24 7; 

In each part, sketch the vectors z\,Z 2 ,z\ | Z 2 > and z\ — zi. 

6 . 

(a) z\ = 3 I i> Z 2 = 1 4 4; 

(b) = _ 2 -4 2b Z2 = 4 I 5; 

In each part, sketch the vectors z and kz. 

7. 

(a) 7 = 1 4- b k = 2 

(b) z= —2 — Abk= -2 

(c) z = 4 + 6i,k=-^ 

In each part, find real numbers and jK that satisfy the equation. 

8 . 

(a) k\i 4- .^2(4 + i) = 3 — 2; 

(b) k\{2 I 3;) 4 k 2 (\ -4i) =7 I 5; 

In each part, find ziz?, _2, and _2. 

9 . ‘1 *9 

(a) z\ — 3;» — 1 — i 


(b) zj=4 -I- 6;> = 2 — 3; 



10 . 


(C) z 1 = 1(2 + 4i),z 2 = £( 1-50 


Given that Z{ = 2 - 5i and Z2 = _ ] _ j, find 


(a) z l ~ z \ z 2 


(b) (z\ 4 3z 2 ) 2 

(c) [z\ 4 (lH z 2 )] 2 


^ lz 2 ~Z\ 


In Exercises 11-18 perform the calculations and express the result in the form a I bi¬ 


ll. 


(14 2j)(4 — 6i) 2 


12 . 


(2-i)(3 + i)C4-2i) 


13. 


(1-30 3 


14. 


i(l f 70 — 3i(4 I 20 


15. 


16. 


-|2 


C2+o(i + |i). 


(^2 I 0 “*l/2(l 4 ^2i) 


17. 


(1 +i4i 2 I z 3 ) 


100 


18. 


(3 — 2i) 2 — (3 4 20 2 


19. 


Let 


A = 


1 i 
-i 3 


5 = 


2 24 ! 

2-i 4 


Find 



( a ) A + 3 iB 

(b) BA 

(c) AB 

( d ) B 2 - A 2 


Let 



Find 


(a) A(BC ) 

(b) (BC)A 

(c) (CA)B 2 

(d) (\+i)(AB) + (3-4i)A 

Show that 

21 . 

(a) Im (iz) = Re (z) 

(b) Re (iz) = — Im (z) 

In each part, solve the equation by the quadratic formula and check your results by substituting the solutions into the given 
22. equation. 

(a) z 2 + 2z + 2 = 0 

(b) z 2 — z -I- 1 = 0 


23 . 


(a) Show that if n is a positive integer, then the only possible values for i H are 1,-1, i, and 



(b) Find;2509. 


Prove: If Z] z 2 = 0» then Zl = o or ?2 = 0- 

24. 


Use the result of Exercise 24 to prove: If zz\ = zz 2 and z^O, then z\ =z 2 . 

25. 


Prove that for all complex numbers z\, z 2 , and z 2 , 

26. 


(a) zi 4 z 2 =z 2 4 zi 


(b) zi 4 (z2 I-Z3) = (z 1 4 Z2) + Z3 


Prove that for all complex numbers z\, z 2 , and z 2 , 

27. 


(a) zjzv Z 2 Z 1 


(b) zi (z 2 z 3 ) = (ziz 2 )z3 


Prove that Z \ (z 2 F z 2 ) =z\z 2 f ziz 3 for all complex numbers z\,z 2 , and Z3. 

28. 


0 0 1 

0 1 0 

10 0 ’ 

0 0 0 

0 1 0 

0 0-1 

0 0 0 

-10 0 


(a) Prove that j3 2 = c? x = aj = a$ = I- 

(b) Two matrices A and B are called anticommutative if AB — — BA- Prove that any two distinct Dirac matrices are 
anticommutative. 


29. 


In quantum mechanics the Dirac matrices are 


.3 = 


a y = 


0 
0 

-1 

0 

0 0 
0 i 
-i 0 
0 0 


CTr = 


Q'rr — 


Describe the set of all complex numbers z = a 4 hi such that a 1 | b 2 = 1- Show that if z\, z 2 are such numbers, then so is 
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10.2 

DIVISION OF COMPLEX 
NUMBERS 


In the last section we defined multiplication of complex numbers. In this 
section we shall define division of complex numbers as the inverse of 
multiplication. 


We begin with some preliminary ideas. 


Complex Conjugates 


If z = a 4 hi is any complex number, then the complex conjugate of z (also called the conjugate of z) is denoted by the 
symbol z (read “ z bar” or “ z conjugate”) and is defined by 

z = a — hi 

In words, z is obtained by reversing the sign of the imaginary part of z. Geometrically, z is the reflection of z about the real 
axis (Figure 10.2.1). 



Figure 10.2.1 

The conjugate of a complex number. 


Examples of Conjugates 


z = 3 + 2i 

z = 3 — 2i 

z= —4 — 2 i 

z= — 4 + 2i 

z = i 

z= —i 

z — 4 

z — 4 


Remark The last line in Example 1 illustrates the fact that a real number is the same as its conjugate. More precisely, it can 
be shown (Exercise 22) that z = z if and only if z is a real number. 


If a complex number z is viewed as a vector in pp, then the norm or length of the vector is called the modulus of z. More 
precisely: 








DEFINITION 


The modulus of a complex number z = a 4 bi, denoted by |z|, is defined by 

M = i + (i) 


If £ = 0, then z = a is a real number, and 

|z| = j/ fl 2 + 0 2 = t/^=M 

so the modulus of a real number is simply its absolute value. Thus the modulus of z is also called the absolute value of z. 



Paul Adrien Maurice Dirac (1902-1984) was a British theoretical physicist who devised a new form of quantum 
mechanics and a theory that predicted electron spin and the existence of a fundamental atomic particle called a positron. 
He received the Nobel Prize for physics in 1933 and the medal of the Royal Society in 1939. 


Modulus of a Complex Number 

Find |z| if z = 3 _ 4j. 

Solution 

From 1, with a = 3 and b = - 4, |z| = 3) 2 q. (-4) 2 = ^25 = 5- 

The following theorem establishes a basic relationship between z and |z|. 


♦ 


THEOREM 10.2.1 












For any complex number z, 


C-l 

±± 

II 

'S3 


If z = a +bi, then 


zz = (a 4 bi ) (a — bi) = eP" — obi 4 bai — b 2 i 2 = a 2 -h b 2 = |z| 2 

■ 

Division of Complex Numbers 


We now turn to the division of complex numbers. Our objective is to define division as the inverse of multiplication. Thus, 
if z 2 * (h then our definition of z = z\! z 2 should be such that 

1— L 

II 

N 

to 

M 

(2) 

Our procedure will be to prove that 2 has a unique solution for z if z 2 * 0> and then to define Z \ 
with real numbers, division by zero is not allowed. 

/ z 2 to be this value of z. As 

THEOREM 10.2.2 


If z 2 ^ 0> then Equation 2 has a unique solution, which is 


z = 1 z \ z 2 

(3) 


Let z — x 4 - iy, z\ — x i 4 - iy\, and z 2 — x 2 + iy 2 Then 2 can be written as 


*1 + iy i - (*2 + iyi)(x +&) 


or 


*1 4- iy 1 = (*2*-727) + i(y 2 X 4 * 27 ) 


or, on equating real and imaginary parts, 


x 2 x-yzy = x\ 
72*-I-*27=71 



-72' 

~x " 



72 

X2 

7 


71 


Since z 2 = x 2 + u ’2 * 0, it follows that x 2 and y 2 are not both zero, so 


*2 

72 


-72 

x 2 


= *2 +72 


( 4 ) 


Thus, by Cramer's rule (Theorem 2.1.4), system 4 has the unique solution 














X = 


*1 

-yi 


7 1 

* 2 

*1*2 y yiy2 *1*2 yiy2 

*2 

-y 2 

2,2 , ,2 

*2+72 hi 

y 2 

*2 




x 2 x \ 

yi 71 

. 71 * 2-*172 _ 71 * 2-*172 


x 2 -y2 

y 2 x 2 

2,2 , ,2 

*2 d 72 hi 


Therefore, 

Z = x + iy = — 1— [ (X i x 2 4 y 172) T i O l *2 - * 172) ] 

hi 

= —^(*1 + * 7 l )(*2 “* 72 ) = ~^ Y Z \ Z 2 

M M 


Thus, for z 2 ^ 0> we define 


5L = 
z 2 


- X~ z i z 2 

hi 


(5) 


Remark To remember this formula, multiply the numerator and denominator of Zl j Z2 by Z 2 : 


£L _ _ 

z 2 - 2 h ^ 2 | 2 ‘ 


- 7 * 1*2 

hi 2 


Quotient in the Form a + bi 


Express 

3 + 4; 

1 - 2 ; 

in the form a f bi- 


Solution 


From 5 with Zl = 3 4 . 4 * and Z2 = 1 _ 2 i, 


3 I 4; 

1 - 2 ; 


=--— t- (3 4 4;) (1 — 2;) = ^- (3 4 4;) (1 4 2;) 

| 1 - 2;| 2 5 

= 1(_5 + 10 ;) = - 1 + 2 ; 


Alternative Solution 


As in the remark above, multiply numerator and denominator by the conjugate of the denominator: 

3 + 4; _ 3 + 4; 14 2; _ — 5 4 10; _ _ 1 . 

1-2; “ 1-2; ' 142; “ 5 

♦ 





























Systems of linear equations with complex coefficients arise in various applications. Without going into detail, we note that 
all the results about linear systems studied in Chapters 1 and 2 carry over without change to systems with complex 
coefficients. Note, however, that a few results studied in other chapters will change for complex matrices. 


A Linear System with Complex Coefficients 


Use Cramer's rule to solve 

ix 4- 2y — \ — 2i 
4x - iy = - 1 f 3 i 


Solution 


X = 


1-2 j 

2 

-1 + 3* 

— * 


* 2 



4 -* 



y = 


i 1—2 i 

4 -1 + 3 * 


i 2 
4 -i 


(-0(1-20-2(-l + 3Q _ -7* 
*(-0-2(4) -7 


(0( 1 I 30-4(1-20 -7 l 7* _ 1 . 

*(-0-2(4) -7 


Thus the solution is x = j, y = l _ j. 


We conclude this section by listing some properties of the complex conjugate that will be useful in later sections. 


THEOREM 10.2.3 

Properties of the Conjugate 

For any complex numbers z, z\, and Z 2 - 

(a) Z[Tz 2 =zi 4-Z2 

(b) z\-zi=z\-Si 

(c) Z\Z 2 =Z\Z 2 

( d ) (>i / Z2) =zi iz2 























(e) z=z 


We prove (a) and leave the rest as exercises. 


Proof (a) Let zj = o\ 4- b\i an d 22 — 0,2 +- £ 2 *’ t * ien 


z\ VZ2 = (a 1 4 - o 2 ) 4 - (£1 -I- b 2 )i 
= + ~ (bi+b 2 )i 

- (a 1 -b\i) 4 - (a2-b2i) 

= z\ 4 ^2 


Remark It is possible to extend part (a) of Theorem 10.2.3 to n terms and part (c) to n factors. More precisely, 

z\ I Z 2 I I z M — z\-\-Z 2 -\ -1 -z n 

Z\Z2"Z yl Z\Z2"Z n 


Exercise Set 10.2 



Click here for Just Ask! 


In each part, find z. 

1 . 

(a) z=2 + 7 i 

(b) ^ = -3 -5i 

(c) ?= 

(d ) z= - i 

(e) z = -9 


(f) ?=0 









In each part, find |z|. 


(a) z = i 

(b) z= -1 i 

(c) z= - 3-4i 

(d) z = 1 hi 

(e) z = -8 

(f) z = 0 

Verify that ^2 for 

(a) z = 2-4i 

(b) z = - 3 4 5; 

(c) z = \[2 — \[2i 

Given that = l _ 5 j and Z2 = 3 4 4 j, find 

(a) z\ / z 2 

(b) z\ ! z 2 

(c) z\ / z 2 

(d) (zi / z 2 ) 

( e ) zi/|z 2 | 


(f) |zi /z 2 | 



In each part, find \ j Z - 


5 . 


(a) z = i 

(b) z = 1 - S 

(0 z=-=i 


Given that Zl = 1 4 j and Z2 = ] _ 2 b find 

6 . 



(b) zi — 1 
^2 



(d) £L 

1Z2 


In Exercises 7-14 perform the calculations and express the result in the form a 4. fa. 


2 

8. (1-i)(3+0 


9- (3 I 40 2 


2 I i 

10. 0-34 40 


i/ 34 i 

n * (l-0(1/3-0 
1 

12. 03-20(1+0 



13. (1-0(1-20(1 f 20 

1 — 2; _ 2 I ; 

14. 3 | 4; 5; 

In each part, solve for z. 

15. 

(a) iz = 2 — i 

(b) (4 — 3 i)z = i 

Use Theorem 10.2.3 to prove the following identities: 

16. 

(a) z-h 5; =z — 5; 

(b) iz — —iz 

(c) ; I z _ _ ^ 
i —z 

In each part, sketch the set of points in the complex plane that satisfies the equation. 

17. 

(a) M = 2 

(b) |z-(l -l-0| = l 

(c) |z-;| = |z + ;| 

(d) Im (z | 0 = 3 

In each part, sketch the set of points in the complex plane that satisfies the given condition(s). 

18. 

(a) Y 1 l \- 1 


(b) 1 < |z| < 2 



(c) |2z — 4i| < 1 


(d) |z|<|z + i| 

Given that z = x -\ iy, find 

19. 

(a) Re (iz) 

(b) Im (iz) 

(c) Re (iz) 

(d) Im (iz) 


20 . 

(a) Show that if n is a positive integer, then the only possible values for (1 / i) H are 1,-1, i, and 


(b) Find(l/i) 2509 . 


Hint See Exercise 23(b) of Section 10.1. 


Prove: 

21 . 

( a ) -^-(zd z) = Re (z) 


(f*) ~~ (z — z) = Im (z) 


Prove: z = z if and only if z is a real number. 

22 . 


Given that Zl = x { + i yi and = x 2 + i ^2 * 0, find 

23. 


(a' tz*\ 

Prove: If (z) =z * /hen ™ is either real or pure imaginary. 

24. 


Pro'v^ thfe® |zf = |z]. 


25 . 



Prove: 


26 . 


(a) z\ -z 2 =z\ -z 2 

(b) z\Z2=z\Z2 

( c ) (z i iz 2 ) = zi )Z 2 

(d) 


27. 

(a) Prove that f = (z) 2 . 


(b) (b) Prove that if n is a positive integer, then z n — (z) n . 


(c) Is the result in part (b) true if n is a negative integer? Explain. 

In Exercises 28-31 solve the system of linear equations by Cramer's rule. 

zxi — zx 2 — — 2 

28. ^ 

lx i f *2 — 1 


xi +X 2 = 2 

29. 

Xl —X 2 — h 

x\ + X2 4 X3 = 3 

30. 0 , 0 . 

Xl 4- X2 — X 3 = 2 4 ll 

xi -X2 4 X3 = - 1 

— z 
-2 z 
0 

the system of linear equations by Gauss-Jordan elimination. 


ix i 4 3x2 + 0 + 0*3 = 

31. 

xi I ZX 2 I 3x3 = 

xi+ X 2 + X 3 = 
In Exercises 32 and 33 solve 


32. 


33. 


_1 -l-f 

'*l' 


'O' 

_-l+z -2 

_*2_ 


_0_ 

2 -l-f 

'*f 


'0" 

_ - 1 | z 1 

_*2_ 


_0_ 


34 . 


Solve the following system of linear equations by Gauss-Jordan elimination. 





4 - ix 2 — ix 3 = 0 

— jti 4 - (1 — i)x2 ¥ 2ix2 = 0 

2x\ 4 - ( — 14 2i)x2 — 3ix2 = 0 


In each part, use the formula in Theorem 1.4.5 to compute the inverse of the matrix, and check your result by showing 
35. that AA~ l =A~ l A = I- 


(a) 




(b) 



i 

0 


Let p (x) = aQ y a\x 4 a-jx 1 I - b a n x n be a polynomial for which the coefficients aQ, a\,a2, ..., a H are real. Prove 

36* that if z is a solution of the equation p(z ) = 0 ? then so is z. 

Prove: For any complex number z, \ Re (z) | < [z| and | Irn (z) | < [z|. 

37. 


Prove that 


38. 


Re (z) | 4- | Im (z) \ 




Hint Let z = x 4 iy and use the fact that (^i _ |y’|) 2 > 0- 


In each part, use the method of Example 4 in Section 1.5 to find j4 -1 , and check your result by showing that 
39 ' AA~ l —A~ l A — L 


(a) 1 14 i 0 

A= 0 1 i 

— i 1 — 2 i 2 


(b) 

A = 


i 0 

0 1 

2 — i i 


— i 

- 1 — 4i 
3 


Show that |z — 11 = |z — 11. Discuss the geometric interpretation of the result. 

40 . 




41 . 


(a) If z[ = a\ 4 and Z2 = an I & 2 i> Yi~ z 2\ an d interpret the result geometrically. 

(b) Use part (a) to show that the complex numbers 12, ^ _|_ 2j, and % _|_ Qj are vertices of a right triangle. 


Use Theorem 10.2.3 to show that if the coefficients a, b, and c in a quadratic polynomial are real, then the solutions of 
42. the equation az 2 -\ bz f c — 0 are complex conjugates. What can you conclude if a, b, and c are complex? 
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10.3 

POLAR FORM OF A 
COMPLEX NUMBER 


In this section we shall discuss a way to represent complex numbers using 
trigonometric properties. Our work will lead to an important formula for 
powers of complex numbers and to a method for finding nth roots of complex 
numbers. 


Polar Form 

If z = x 4 - iy is a nonzero complex number, r = |z|, and 0 measures the angle from the positive real axis to the vector z, then, as 
suggested by Figure 10.3.1, 

x — r cos#, y—rsm.9 ^ 

so that z — x + iy can be written as .7 — r ra , ; f) 4. 1T sin 0 or 

z = r(cos 8 + i sin#) 


This is called a polar form of z. 

Argument of a Complex Number 

The angle 0 is called an argument of z and is denoted by 

9 = arg z 

The argument of z is not uniquely determined because we can add or subtract any multiple of 2 tt from 0 to produce another 
value of the argument. However, there is only one value of the argument in radians that satisfies 

-x<e<x 

This is called the principal argument of z and is denoted by 

0 = Arg z 



Figure 10.3.1 


Polar Forms 


Express the following complex numbers in polar form using their principal arguments: 


(a) z = 1 4 - \[3i 








(b) z = - 1 - i 


Solution (a) 

The value of r is 

r=|z| = |/l 2 I C/3) 2 = |/4 = 2 
and since x — 1 and y = \j~ 3 , it follows from 1 that 

1=2 cos# and ^=2 sin# 

so cos # = 1 / 2 and sm q = I 2- The only value of i ) that satisfies these relations and meets the requirement — ;r < # < x is 
B — 7 T / 3( = 60°) (see Figure 10.3.2a). Thus a polar form of z is 

z = 2 ^cos y -I- z sin J 


Solution (b) 


The value of r is 


'■=l*l = l/(-l) 2 +(-l) 2 = l/2 

and since x = _ ] , y = — 1 , it follows from 1 that 

— l = ^cos# and — l = ^sin# 

so cos 9 = —1/^2 and sin# = — 1 / tjl- The only value of 9 that satisfies these relations and meets the requirement 
— 7 r<#< 7 ris 0 = — 3tt / 4 ( = — 135°) (Figure 10.3.2 b). Thus, a polar form of z is 


:=fe 


COS 


— 3tt 


+ i sin 


-3?r 




(M 

Figure 10.3.2 


♦ 


Multiplication and Division Interpreted Geometrically 

We now show how polar forms can be used to give geometric interpretations of multiplication and division of complex 
numbers. Let 


Z 1 =ri(cos#i -Msin#i) and Z 2 = ^(cos #2 + * sin # 2 ) 










Multiplying, we obtain 


z\zo. = nn\ (cosd] cos ~ sinfli sin#?) 4- i(sin0i cos Bo. 4- cos#i sin #2)1 
Recalling the trigonometric identities 

cos (#1 4- 02) = cos9\ cos &2 — sin 9\ sin #2 
sin(#i 4- # 2 ) = sincos $2 I- cos 9\ sin 9'i 

we obtain 


z\Z 2 = r\r 2 [ cos (9\ 4- 8 f) 4- * sin(0i + 82 )] 


which is a polar form of the complex number with modulus r\r 2 and argument + 82 - Thus we have shown that 

fr z 2|= |zi||Z2| 


(3) 

(4) 


and 


arg(ziZ 2 ) - argzi -1 argZ 2 


(Why?) In words, the product of two complex numbers is obtained by multiplying their moduli and adding their arguments 
(Figure 10.3.3). 


We leave it as an exercise to show that if Z 2 * 0> then 


£L = H 
z 2 n 


[cos(0i - 82 ) 


i sin(0i - 82 )] 


(5) 


from which it follows that 


and 


ZL = M 
z 2 M 


ifz2 * 0 


ar g(fJ-) = ar S^l = 

In words, the quotient of two complex numbers is obtained by dividing their moduli and subtracting their arguments (in the 
appropriate order). 



Figure 10.3.3 

The product of two complex numbers. 


A Quotient Using Polar Forms 


Let 

z\ = 1 4- 

Polar forms of these complex numbers are 


and 









(verify) so that from 3, 


and from 5, 


z\ = 2 ^cos y -I- i sin y J and Z 2 = 2 ^cos ^ 4- i sin ^ J 

z,z 2 =4[ cos(f + f)+i«n(f + f)] 

= 4^cos H-z sin-|-J = 4 [0 + z] = 4z 


&= 1 -[“ s (f-f) + -(f-f)] 

= cos|tisin| = -^H ±i 


As a check, we calculate z^ and zi /z 2 directly without using polar forms for z\ and zj. 

ziz 2 = (1 + {k) (|/3 + i) = (y[3 - ft) + (3 + l)z = 4i 
z\ _ 1 + i/a i/3 -i _ (/3 fV3) + (-i + 3i) _ J3 1 . 

z 2 j/3 + i /3-i 4 2 2 ! 

which agrees with our previous results. 


♦ 


The complex number i has a modulus of 1 and an argument of / 2( = 90°), so the product j? has the same modulus as z, but 
its argument is 90° greater than that of z- In short, multiplying z by i rotates z counterclockwise by 90° (Figure 10.3.4). 



Figure 10.3.4 

Multiplying by i rotates z counterclockwise by 90°. 


DeMoivre's Formula 

If n is a positive integer and z = r(cos 9 A- 1 sin 0), then from Formula 3, 

z M = z -z -z-z = r M [cos(04- 9 A -h 9) A-i sin(# 4 - 9 A - 1 - 0)] 


a -factors 


h- terms 


h- terms 


or 


z H = (cos n9 A- i sin n9) 

Moreover, 6 also holds for negative integers if z 5 * 0 (see Exercise 23). 

In the special case where r — 1, we have z = cos 9 A -1 sin A so 6 becomes 

(cos#H i sintf ) n = cos n9 A isvanO 

which is called DeMoivre’s formula. Although we derived 7 assuming n to be a positive integer, it will be shown in the 
exercises that this formula is valid for all integers n. 


( 6 ) 


(7) 


Finding nth Roots 










We now show how DeMoivre's formula can be used to obtain roots of complex numbers, if n is a positive integer and z is any 
complex number, then we define an nth root of z to be any complex number w that satisfies the equation 


= z 


( 8 ) 


We denote an nth root of z by If z * 0> then we can derive formulas for the nth roots of z as follows. Let 

w = p(cosa +j sinct) and z — r(cos9 + i sintf) 

If we assume that w satisfies 8, then it follows from 6 that 

p n (cos na 4- i sin na) = r(cos9 + ism9) 


Comparing the moduli of the two sides, we see that p n = r or 

P = n \Tr 

where denotes the real positive nth root of r. Moreover, in order to have the equalities cos net = cos 9 and sin net = sin# in 9, 
the angles n a and 9 must either be equal or differ by a multiple of 2i r- That is, 


not. = 9 4- 2 k~ or 


9 , 2kx 

ot = — + -. 

n n 


Thus the values of w = p(cos a + i sin cl) that satisfy 8 are given by 

="f 


W : 


cos (— 1 

2 k~ 1 

) + i sin I — 1 

2 kn ) 

\* 

n ^ 

1 \* 

» JJ 


k = 0, 11, _L 2, 


i = 0. ± 1, d 2 ,... 


Although there are infinitely many values of k, it can be shown (see Exercise 16) that k — 0. 1,2, 1 produce distinct 

values of w satisfying 8 but all other choices of k yield duplicates of these. Therefore, there are exactly n different nth roots of 
z = r(cos 9 4 i sin 9 ), and these are given by 



* = 0 , 1 , 2 ,— 1 


( 10 ) 



Abraham DeMoivre (1667-1754) was a French mathematician who made important contributions to probability, statistics, 
and trigonometry. He developed the concept of statistically independent events, wrote a major and influential treatise on 
probability, and helped transform trigonometry from a branch of geometry into a branch of analysis through his use of 
complex numbers. In spite of his important work, he barely managed to eke out a living as a tutor and a consultant on 
gambling and insurance. 











Cube Roots of a Complex Number 


Find all cube roots of -8. 


Solution 


Since -8 lies on the negative real axis, we can use 9 = tt as an argument. Moreover, r = |z| = | — 8| = 8, so a polar form of -8 is 

— 8 = 8 (cos 7T 4 i sin ~) 

From 10 with n = 3, it follows that 

(_8) 1/3 = 3 /s cosf^4 ^-l + isinf£ + , *=0,1,2 


cos -\ 

2fcr ' 

1 1 i sui [ ~ 4 

2kV 

L 13 

3 , 

1 \3 

3 }\ 


Thus the cube roots of -8 are 


2^cos f ; sin-jJ = 2 




2(cos tt -I- i sin ~) = 2( — 1) = — 2 

H 


2cos(^- I = = 


2 2 


14 fk 


1 - i 


As shown in Figure 10.3.5, the three cube roots of -8 obtained in Example 3 are equally spaced k 13 radians ( — 120°) apart 
around the circle of radius 2 centered at the origin. This is not accidental. In general, it follows from Formula 10 that the nth 
roots of z lie on the circle of radius n \[r{ = yfjzj) and are equally spaced 2~! n radians apart. (Can you see why?) Thus, once 
one nth root of z is found, the remaining n — \ roots can be generated by rotating this root successively through increments of 
2 k i n radians. 



Figure 10.3.5 

The cube roots of -8. 


Fourth Roots of a Complex Number 


Find all fourth roots of 1. 


Solution 


We could apply Formula 10. Instead, we observe that w = 1 is one fourth root of 1, so the remaining three roots can be 












generated by rotating this root through increments of 2z! 4 = tt / 2 radians ( = 90°)• From Figure 10.3.6, we see that the fourth 
roots of 1 are 

1, i, -1, -i 


i ' 

i 



Figure 10.3.6 

The fourth roots of 1. 


Complex Exponents 


We conclude this section with some comments on notation. 

In more detailed studies of complex numbers, complex exponents are defined, and it is shown that 

cos# + i sin0 = e 3 ^ (11) 

where e is an irrational real number given approximately by e ~ 2.71828-• •• (For readers who have studied calculus, a proof of 
this result is given in Exercise 18.) 

It follows from 11 that the polar form 


can be written more briefly as 


z — r(cos 9 i sin#) 


z = 



( 12 ) 


EXAM Expressing a Complex Number in Form 12 

In Example 1 it was shown that 

1 I- ^3i = 2^cos y + i sin-jJ 

From 12 this can also be written as 

14 fii = 2e ! ' T/3 

♦ 

It can be proved that complex exponents follow the same laws as real exponents, so if 

z\ — and Z 2 — 


are nonzero complex numbers, then 







201 +3t?-. 2 ft? I +£?■?) 

z\Z2 = r\r2e 1 A = r\r2& K 1 z 

^L — LL JOi-*h - CL J(Pi^2) 
Z2 ~ n e r 2 


But these are just Formulas 3 and 5 in a different notation. 


We conclude this section with a useful formula for z in polar notation. If 

z = re ll ’ > = r(cos9 + i sin#) 


then 


z = r(cos9 — i sm.9) 


Recalling the trigonometric identities 
we can rewrite 13 as 

or, equivalently, 


sin( —&)= — sin# and 
z = r [ cos ( — 0 ) + ! sin( 


cos( — 9) = cos 9 
3)] = re i( ~ 0) 


z = re 


-iO 


In the special case where r = ], the polar form of z is z = e lf \ and 14 yields the formula 


JO »~ id 
e =e 


(13) 


(14) 


(15) 


Exercise Set 10.3 



Click here for Just Ask! 


In each part, find the principal argument of z- 

1 . 


(a) z=l 


(b) z = i 


(c) z= -i 

(d) z = 1 + i 


(e) z=-\ + fii 


(f) z = 1 J 




2 . 


In each part, find the value of Q = arg(l — that satisfies the given condition. 


(a) 0<e<2% 

(b) — ^ < 9 < it 

(c) _ E < $ < He. 

6 ~ 6 

In each part, express the complex number in polar form using its principal argument. 

(a) 

(b) -4 

(c) 5-1 5 i 

(d) —6 f 6^3; 

(e) -3-3; 

(f) 2^3-2 i 

Given that Zl = 2(cos tt / 4 4 - i sin k / 4) and Z2 = 3(cos tt / 6 + i sin it / 6 ), find a polar form of 

(a) z l z 2 

(b) £1 
z 2 

(c) £2. 

*1 

(d) Z J_ 


5 . 


Express Z \ =;, Z2 = 1 — and z ^ _ ^3 | j in polar form, and use your results to find Z \zi Izy Check your results by 
performing the calculations without using polar forms. 



Use Formula 6 to find 


6 . 


(a) (1 |-!) 12 


(b) 



-6 


<c) h ) 7 


< d > (l-i/f)’ 10 


In each part, find all the roots and sketch them as vectors in the complex plane. 

7. 

(a) (— i) 1/2 

(1 + ^) 1/2 

(c) (-27) 1/3 

(d) (0 1/3 

(e) (-1) 1/4 

(f) (-84 8^3i) 1M 


Use the method of Example 4 to find all cube roots of 1. 

8 . 


Use the method of Example 4 to find all sixth roots of 1. 

9. 


Find all square roots of 1 | j and express your results in polar form. 

10 . 


Find all solutions of the equation = Q. 


11 . 



Find all solutions of the equation z 4 _8 = 0 and use your results to factor z 4 -|- 8 into two quadratic factors with real 

1 2 

coefficients. 

It was shown in the text that multiplying z by i rotates z counterclockwise by 90°. What is the geometric effect of dividing 

13. z by /? 

In each part, use 6 to calculate the given power. 

14. 

(a) (1 4 0 8 

( — 2^3 f2i) -9 

In each part, find R e (z) and Im ( z ). 

15» 

(a) z = 3e !T 

(b) z =3e~ in 

(c) F= fee™' 2 

(d) z= -3 e - 2 ^' 

16. 

(a) Show that the values of z 1/m in Formula 10 are all different. 

(b) Show that integer values of k other than fr = 0, 1,2, — 1 produce values of z 1/m that are duplicates of those in 

Formula 10. 

Show that Formula 7 is valid if n = 0 or n is a negative integer. 

17. 

18. (For Readers Who Have Studied Calculus) To prove Formula 11, recall that the Maclaurin series for g x 

X x 2 x n 

e x =I m 4- + - + ^ + - 
2 ! «! 


(a) By substituting x = $ in this series and simplifying, show that 



JO 


= 1 - 


9± 92 

2! 4! 




2i i i 

3! 5! 7! 


(b) Use the result in part (a) to obtain Formula 11. 


Derive Formula 5. 

19. 


20 . 


When n — 2 and n — 3, Equation 7 gives 

(cos 9 4 i sin 9) 2 = cos 20 4 i sin 20 
3 

(cos 0 4 ! sin#) = cos 30 [ i sin 30 

Use these two equations to obtain trigonometric identities for C os 20, sin 20> cos 3 9, and sin 3 Q. 


Use Formula 11 to show that 


21 ' J° i 

cos 9 = ——- 


and 


JO »-i0 
sin 9 = - e ~- 


2 i 


Show that if (tat \- bi ) 3 = 8, then a 2 + b 2 = 4- 

22 . 


Show that Formula 6 is valid for negative integer exponents if z * 0- 

23. 
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10.4 

COMPLEX VECTOR 
SPACES 


In this section we shall develop the basic properties of vector spaces with 
complex scalars and discuss some of the ways in which they differ from real 
vector spaces. However, before going farther, the reader should review the 
vector space axioms given in Section 5.1. 


Basic Properties 

Recall that a vector space in which the scalars are allowed to be complex numbers is called a complex vector space. Linear 
combinations of vectors in a complex vector space are defined exactly as in a real vector space except that the scalars are 
allowed to be complex numbers. More precisely, a vector w is called a linear combination of the vectors of vj, V 2 , . v r if 
w can be expressed in the form 

W = £iVi -I- ^2 v 2 + "■ + 

where •••, k r are complex numbers. 

The notions of linear independence, spanning, basis, dimension, and subspace carry over without change to complex vector 
spaces, and the theorems developed in Chapter 5 continue to hold with R n changed to C n - 


Among the real vector spaces the most important one is R n , the space of n-tuples of real numbers, with addition and scalar 
multiplication performed coordinatewise. Among the complex vector spaces the most important one is C n , the space of 
n-tuples of complex numbers, with addition and scalar multiplication performed coordinatewise. A vector u in C n can be 
written either in vector notation, 

U= (hi, U 2 , U n ) 


or in matrix notation, 


where 


Hi 

u 2 

u = 

H h 

u\=a\+b\i, U 2 =ct 2 +i> 2 i, ■■■, it n = a„-\-b„i 


Vector Addition and Scalar Multiplication 


If 

then 

and 


u = (z, 1 + — 2) and v = (2 + i, 1 — i, 3 + 2i) 


u I- v = (z, 1 f — 2) + (2 -f i, 1 —3 4- 2i) = (2 4- 2i, 2, 1 f 2i) 


ju = i(i, 1 + i, — 2) = (i 2 , i i 2 , — 2i) = ( — 1, — 1 f i, — 2i) 


In C M as in R n , the vectors 


*1 = (1. 0, 0.0), e 2 = (0. 1. 0.0). = (0, 0, 0.1) 









form a basis. It is called the standard basis for C H . Since there are n vectors in this basis, C n is an n-dimensional vector 
space. 


Do not confuse the complex number j = yj — \ with the vector i = (1, 0, 0) from the standard basis for p} (see 
Example 3, Section 3.4). The complex number i will always be set in lightface type and the vector i in boldface. 


Complex M mn 


In Example 3 of Section 5.1 we defined the vector space M mn of m x n matrices with real entries. The complex analog of this 
space is the vector space of mxn matrices with complex entries and the operations of matrix addition and scalar 
multiplication. We refer to this space as complex M myi - 

♦ 


Complex-Valued Function of a Real Variable 


If f j (x) and f 2 (x) are real-valued functions of the real variable x, then the expression 

/OO = ./iOO -K/OOO 

is called a complex-valued function of the real variable x. Two examples are 

/ (x) = 2x 4-zx and g(x) = 2 sinx -I- i cos x ( 1 ) 

Let Vbe the set of all complex-valued functions that are defined on the entire line. Iff = f j (x) \ if 2 (x) and 
£ = gj (x) + ig 2 (x) are two such functions and k is any complex number, then we define the sum function f + a and the 
scalar multiple kf by 

(f + g)00 - [/i00 +gi(^)] +;[/2(^) +g2(*)] 

(kf)(x) — kf i(x) +ikf 2 (x) 

For example, iff = f (*) and s = g(^) are the functions in 1, then 

(f -I- g) (x) = ( 2x -f 2 sinx) -I- i(x 3 4- cos x) 

(if) (x) = 2xj -I i 2 x 3 = — x 3 4- 2xi 

((1 4- i) g)(x) = (1 4-0(2 sinx 4- i cosx) = (2 sinx — cosx) 4 -i(2 sinx 4- cosx) 

It can be shown that V together with the stated operations is a complex vector space. It is the complex analog of the vector 
space F(^ oo , oo ) of real-valued functions discussed in Example 4 of Section 5.1. 

♦ 


Complex C( — oo , oo ) 


Calculus Required 

If / (x) = /1 (x) 4- if 2 (x) is a complex-valued function of the real variable x, then/is said to be continuous if f j (*) and 









f 2 (;r) are continuous. We leave it as an exercise to show that the set of all continuous complex-valued functions of a real 
variable x is a subspace of the vector space of all complex-valued functions of x. This space is the complex analog of the 
vector space C{— do , do ) discussed in Example 6 of Section 5.2 and is called complex C{— do , oo )• A closely related 
example is complex C[a, b], the vector space of all complex-valued functions that are continuous on the closed interval 
[a,b]. 


Recall that in R n the Euclidean inner product of two vectors 

u= Oi, U'l, ..,a M ) and v = (vi, v„) 


was defined as 


U-Y = H1V1 +U2V2+" + U n Vyi) 


and the Euclidean norm (or length) of u as 


( 2 ) 


INI = (" ■ ll ) 1/2 = (/a?+ a| + - + aj (3) 

Unfortunately, these definitions are not appropriate for vectors in C n - For example, if 3 were applied to the vector u = (i, 1) 
in c 1 , we would obtain 

||u|| = |/i 2 +l = /0 = 0 

so u would be a nonzero vector with zero length—a situation that is clearly unsatisfactory. 


To extend the notions of norm, distance, and angle to C M properly, we must modify the inner product slightly. 


DEFINITION 


If u = (hi, U 2 , u n ) and v = (vi, V 2 , ..., v M ) are vectors in C M , then their complex Euclidean inner product u ■ v is 
defined by 

U-V = H1V1 -I- U'p?2 H-f 

where v 1? V 2 > •••, v H are the conjugates of vi, V 2 , •••, v H . 


Remark Observe that the Euclidean inner product of vectors in C H is a complex number, whereas the Euclidean inner 
product of vectors in R M is a real number. 


Complex Inner Product 


The complex Euclidean inner product of the vectors 

u = ( — i, 2, 1 -I- 3i) and y= (1 — i, 0, 1 -I- 3i) 

u-Y=(-0(1^0 4- (2)(0) + (1 + 30(1 + 30 
= (-0(l+0 + (2)(0) +■ (14 30(1-30 
= -i-i 2 + 1 -9i 2 = 11 -i 


IS 













Theorem 4.1.2 listed the four main properties of the Euclidean inner product on R n . The following theorem is the 
corresponding result for the complex Euclidean inner product on C H . 


THEOREM 10.4.1 


Properties of the Complex Inner Product 

If u, v, and w are vectors in C n , and k is any complex number, then 


(a) u-v = v-u 

(b) (u 4- v) ■ w — u - w 4 v - w 

(c) (£u) - v = Ar(u - v) 

(d) y - v > 0- Further, y - v = 0 if and only if y — 0. 


Note the difference between part (a) of this theorem and part (a) of Theorem 4.1.2. We will prove parts (a) and ( d) and leave 
the rest as exercises. 


roof (a) Let u = (u lf u 2 , u n ) and v = (vi, v 2 ,..., v„)- Then 


and 

so 


u-v = uivi 4- u-pn H-1 

v-u = viui 4- v 2 £?2 4- -4- v„u H 


V - U = VlUl 

1 v 2 u 2 4-" 




= viW! 

1 v 2 u 2 + " 

■4- v n u„ 

[Theorem 10.2.2 

K (a) and {c)\ 

= vi«i 

= «ivi 

= U - V 

1 V2U2 + - 

1 U2V2 + - 

•4 

■4 -u n v„ 

[Theorem 10.2.2 

pait (e)] 


Proof (d) 

v ■ v = v!vi 4- v 2 v 2 4- -4- V„V H = |v 1 1 2 4- |v 2 | 2 4- -4- |v„| 2 > 0 

Moreover, equality holds if and only if | V1 1 = |v 2 | = - = |v„| = 0. But this is true if and only if 
vi = v 2 = ■■■ = v H = o; that is, it is true if and only if v = o. 


Remark We leave it as an exercise to prove that 






u ■ ( kv) = &(u - v) 

for vectors in C n . Compare this to the corresponding formula 

u - (kv) = £(u - v) 

for vectors in 

Norm and Distance in C !> 

By analogy with 3, we define the Euclidean norm (or Euclidean length ) of a vector u = (u ], u% u„) in C n by 

INI = («-«) 1/2 = ^| ui | 2 + | w 2 | 2 + - + | a H | 2 

and we define the Euclidean distance between the points u = (ui, uj,u n ) and v = (vi, v 2 , v M ) by 

d = (u, v) = ||u - v|| = |/|ai -vi| 2 + |^2 — v 2 | 2 -I-h |u H -v„| 2 


EXAMPLE 6 Norm and Distance 


If u = (j, 1 4- i, 3) and v = (1 — i, 2, 4i), then 

||u|| = | 1 + i |2 + | 3 |2 = /TT2T9 = /l2 = 2/3 

and 


^|j- (1 -! 

)| 2 +|(l +0 

— 2 

| 2 + |3 

— 4i| 2 

f| —1 + 2 j| 

l 2 +l 

— 1 4- i\ 

| 2 + |3-4i| 

2 


= ^5+ 2 + 25= /32 = 4/2 


The complex vector space C M with norm and inner product defined above is called complex Euclidean n-space. 


Exercise Set 10.4 



Click here for Just Ask! 


Let u = (2 i, 0, — 1, 3); v= ( — i, i, 1 + *, — 1)> and w= (1 + i, —i, — 1 f 2i, 0)- Find 

(a) U“Y 

(b) iv 4- ?.v! 


(c) - w T v 














(d) 3(11 — (1 -I i)v) 


(e) - *v 4- 2hv 

(f) 2v — (u 4- w) 

Let u, v, and w be the vectors in Exercise 1. Find the vector x that satisfies u — v 4 ix = 2i\ 4- w- 

2 . 

Let Ul = (1 - i, i, 0), u 2 = (2i, 1 ¥i, 1), and ll3 = (0, 2z, 2 -z). Find scalars c\, C 2 , and c 3 such that 
ciui 4- C2 u 2 + C 3 U 3 = ( — 3 f- i, 3 4 2z, 3 — 4i)- 

Show that there do not exist scalars c\,C 2 , and c 3 such that 
ci (z, 2 — i, 2 + z) 4- 4- i, — 2z, 2) 4- c 3 (3, i, 6 4- z) = (z, i, i) 

Find the Euclidean norm of v if 

5. 

(a) v= (1, z) 

(b) v= (1 4 i, 3z, 1) 

(c) v = (2z, 0, 2z I 1,-1) 

(d) V = ( — z, z, z, 3 , 3 4 4 z) 


6 . 


Let u = (3^ 0, — z), v = (0, 3 4 4 i, — 2and w = (1 4 z ; 2z, 0)- Find 


(a) l|n + v|| 

(b) INI + IMI 

(c) II-ra|| 4-* INI 

(d) || 3u — 5v 4 w|| 


(e) 


1 


w 





Show that if v is a nonzero vector in C M , then (1 / ||v||)v has Euclidean norm 1. 


7. 


Find all scalars k such that ||tej| = 1 , where v = ( 3 i Ai)- 

8 . 


Find the Euclidean inner product u . v if 

9. 


(a) u = ( — i, 3i), v = (3i, 2 i) 


(b) u = (3 — Ai, 2 f i, — 6i )> v = (1 - 1 - i, 2 — i, 4) 


( c ) u = (1 — i, 1 4- i, 2i, 3 ), v = (4 4- Si, — 5i, — 1 I i, i) 

In Exercises 10 and 11 a set of objects is given, together with operations of addition and scalar multiplication. Determine 
which sets are complex vector spaces under the given operations. For those that are not, list all axioms that fail to hold. 

The set of all triples of complex numbers (z\,Z 2 , 23 ) w hh the operations 
10 * (21,22.23) + {z[,z ! 2 ,z ! 2 ) = {z\ -4 z[,z 2 -\ z' 2 ,Z2 I z 2 ) 

and 

kiz\,Z2,zi) = (tei, kz2, kzi) 

The set of all complex 2x2 matrices of the form 

11. ' r nl 

z 0 
0 z 

with the standard matrix operations of addition and scalar multiplication. 

Use Theorem 5.2.1 to determine which of the following sets are subspaces of f 7 3 : 

12 . 


(a) all vectors of the form ( z , 0 , 0 ) 


(b) all vectors of the form 


(c) all vectors of the form (z\,Z 2 , 23 )’ where z 2 =z\ I Z' 2 


(d) all vectors of the form {z\,z 2 , 23 ), where z 3 = Z \ + Z 2 4 : 


Let T : c 3 


13 . 


C 3 be a linear operator defined by T’(x) = where 



i 


A = 


1 

0 


— i — 1 

— i 1 -I - i 
1 -i 1 


Find the kernel and nullity of T. 


Use Theorem 5.2.1 to determine which of the following are subspaces of complex Min'- 

14. 


(a) All complex matrices of the form 


where z\ and ?2 are real. 

(b) All complex matrices of the form 


where Zl i z 4 = 0- 


*1 z 2 
z 2 Z A 


Z\ z 2 
z 2 Z A 


(c) 


— T — 

All 2x2 complex matrices A such that (^4) = A, where A is the matrix whose entries are the conjugates of the 

corresponding entries in A. 


15. 


Use Theorem 5.2.1 to determine which of the following are subspaces of the vector space of complex-valued functions 
of the real variable x: 

(a) all / such that y (l) = 0 

(b) all/such that / (0) =: 

(c) all/suchthat/( — x) =/(x) 

(d) all /of the form ^ _|_ k 2 S ix - where and are complex numbers 

Which of the following are linear combinations of u = (/ and v = (2i, 4i, 0) 

(a) (3i.3i.3i) 

(b) (4i, 6i) 

(c) (i, 5i, 6i ) 


16 . 


(d) (0,0,0) 


Express the following as linear combinations of u = (1, 0, — z), v = (1 I- z, 1, 1 — 2i), and w — ( 0 , j, 2)- 

17. 

(a) ( 1 , 1 , 1 ) 

(b) (i, 0 , -0 

(c) (0,0,0) 

(d) (2 — z, 1 , 1 I z) 

In each part, determine whether the given vectors span f_ 7 3 - 

18. 

(a) vi — (z, i, i), v? — (2z, 2z, 0), V 3 — (3z, 0, 0) 

(b) V1 = (1 f z, 2 -z ,3 + z)>v 2 = (2 + 3 z, 0 , 1 -z) 

(c) vi = (1, 0, -i)> v 2 = (1 4 z, 1, 1 -2z), v 3 = (0,z, 2) 

(d) vi = (U, 0 ),v 2 = ( 0 , -z, l),v 3 = (l, 0 , 1 ) 

Determine which of the following lie in the space spanned by 
19 ‘ f=e ix and g = e~ ix 

(a) cosx 

(b) sin* 

(c) cos x 4- 3z sinx 

Explain why the following are linearly dependent sets of vectors. (Solve this problem by inspection.) 

20 . 

(a) 111 = (1 — z, 0 an< ^ 112 = O + *- — 1 ) i° C 2 






(b) U1 = (1, -i), ii2 = (2 4- i, - 1)’ U 3 = (4, 0) in c 2 


(c) 



3 i 
0 


and B = 


1 3 

2 0 


in complex M 22 


Which of the following sets of vectors in c 2 are linearly independent? 

21 . 


(a) U1 = (1 -J, 1, 0), 112 = (2, 1 + i, 0), 113 = (1 I i, i, 0) 


(b) m = ( 1 , 0 , -i), U 2 = (1 H i, 1 , 1 - 2 i), 113 = ( 0 , i, 2 ) 


(c) ui = (j, 0, 2-i)’ U 2 = (0, 1, 0’ 113 = ( “i, - 1 -4i, 3) 


Given the vectors v = [a, b I ci~\ and w = [b —ci, a], for what real values of a, b, and c are the vectors v and w linearly 

22 . dependent? 


Let V be the vector space of all complex-valued functions of the real variable x Show that the following vectors are 

23. linearly dependent. 

2 2 2 2 
f = 3H 3i cos 2x, g = sin x 4- : cos x, h=cos x— j sin x 

Explain why the following sets of vectors are not bases for the indicated vector spaces. (Solve this problem by 

24. inspection.) 


(a) U1 = (j r 20 ’ U 2 = ( 0 , 30’ 113 = ( 1 , 70 for C 2 

(b) ui = ( — 1 f i, 0 , 2 — 0 ’ U 2 = ( 1 , — i, 1+0 f° r C 2 
Which of the following sets of vectors are bases for 


(a) (2 j, -0’ (40 0) 

(b) (1 f i, 1 ), (14 i, 0 


(c) ( 0 , 0 ), (1 +j, 1 -0 



(d) (2 — 3i, i), (3 1-2;, - 1) 


Which of the following sets of vectors are bases for c 3 ? 

26. 


(a) (i, 0, 0), (i,!, 0>(i,!, 0 

(b) (1, 0, -i), (1 +i, 1, 1-2;), (0,!, 2) 


(c) 


a 0, 2-0. (0,1,0, ( —i, -1-4!, 3) 


(d) (1,0,!)’ (2 —i, 1,2-1 0* C0.3i.3i) 


In Exercises 27-30, determine the dimension of and a basis for the solution space of the system. 

*1 + (1 + j )*2 = 0 

^’ (1 — 0*1 4- 2x2 = 0 

2xi — (1 I- 0*2 = 0 

28. / 1 A 

( — 1 f 0*1 I *2 = 0 

xi -I- (2 — 0*2 = 0 

29. , n 

X 2 f 3!X 3 = 0 
!xj 4 (2 I 20^2-1 3ix3 = 0 

*1 I JX 2 — 2 ix 3 f xa = 0 

30. 

ixi 4 3x2 4- 4x3 — 21 x 4 = 0 

Prove: If u and v are vectors in complex Euclidean n-space, then 

u - (kv) = &(u - v) 


32. 

(a) Prove part ( b ) of Theorem 10.4.1. 

(b) Prove part (c) of Theorem 10.4.1. 


Establish the identity 
33 

«-v = ^-||u4 v|| 2 -^-||u-v|| 2 + ^-||u4 iv|| 2 -^||u-!v 
for vectors in complex Euclidean n-space. 


34. (For Readers Who Have Studied Calculus) Prove that complex C( — co , oo ) is a subspace of the vector space of 





complex-valued functions of a real variable. 


Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 




10.5 

COMPLEX INNER 
PRODUCT SPACES 


In Section 6.1 we defined the notion of an inner product on a real vector 
space by using the basic properties of the Euclidean inner product on R ri as 
axioms. In this section we shall define inner products on complex vector 
spaces by using the properties of the Euclidean inner product on C n as 
axioms. 


Complex Inner Product Spaces 

Motivated by Theorem 10.4.1, we make the following definition. 


DEFINITION 


An inner product on a complex vector space Visa function that associates a complex number (u, vj with each pair of 
vectors u and v in V in such a way that the following axioms are satisfied for all vectors u, v, and w in V and all scalars k. 

(a) (h, yJ = (v, uJ 

(b) (u -I v, w} = (u, w} -I- (v, w} 

(c) (An, v) = jfc(u, v) 

(d) (v, v} > 0 and (v, v} = 0 if and only if v = 0 

A complex vector space with an inner product is called a complex inner product space. 

The following additional properties follow immediately from the four inner product axioms: 

(i) (0,v} = (v,0} = 0 

(ii) (*, v 4 w} = (u, v} 4 (u, w) 

(iii) (n, kv) = £(u, v} 


Since only (iii) differs from the corresponding results for real inner products, we will prove it and leave the other proofs as 










(u, £v} = (jfcv, uj 

= k!v, uj 

= k{y, u} 
= kin, vj 


[Axiom 1] 

[Axiom 3] 

[Property of conjugates] 
[Axiom 1] 


Inner Product on C n 


Let u — (ui, U 2 , ..., u n ) and y = (vj, V 2 ,v H ) be vectors in C n - The Euclidean inner product 
(u, v'j — u - v = ujvi 4 - U 2 V 2 H- 1 satisfies all the inner product axioms by Theorem 10.4.1. 

♦ 


Inner Product on Complex M 22 


If 



u 2 

U4 


and 



V2 

v 4 


are any 2 x 2 matrices with complex entries, then the following formula defines a complex inner product on complex M 22 
(verify): 

( U, FJ = u\v\ 4- U 2 V 2 4- U 3 V 3 4- U 4 V 4 


For example, if 


then 


U = 


0 i 
1 1 + i 


and V = 


1 — i 

0 2 i 


{U, V) = (0)(1) 4- i( -i) + (1)(0) + (1 + 0(20 
= (0)(1) -H(0 4- (1)(0) A (1 +0(-20 


— 0 +i 2 + 0 — 2 i — 2 i 2 
= 1 - 2 i 


♦ 


Inner Product on Complex C[a, h] 


Calculus Required 

If / (x) = /1 (x) 4- if 2 (x) is a complex-valued function of the real variable x, and if f j (x) and f 2 (x) are continuous on 
[a,b] then we define 

I f(x)dx=f [f\(x) + if 2 (x)]dx= f fi(x)dx + if f 2 &)dx 
Ja Ja Ja Ja 

In words, the integral of fix) is the integral of the real part off plus i times the integral of the imaginary part off 





















We leave it as an exercise to show that if the functions f = f j (*) + if 2 (x) and g = g\ (V) | ig2(x) are vectors in complex 
C\a, A] then the following formula defines an inner product on complex C\a, ij: 

( f , Z)=j 

■h 


[/lW + 1/2(1)] [glOO H ig 2 (x)]dx 


-1: 

-I 


[/lW +1/2W] [glOO -ig 2 (x)]dx 

b rb 

L/lCOglO) I f 2 &)g 2 (x)]dx+i / [f 2 &)gl(x) ~ f l(*)g 2 &)]dx 


In complex inner product spaces, as in real inner product spaces, the norm (or length ) of a vector u is defined by 

IMI = {«>«} 1/2 

and the distance between two vectors u and v is defined by 

d (u, v) = ||u — v|| 

It can be shown that with these definitions, Theorems 6.2.2 and 6.2.3 remain true in complex inner product spaces (Exercise 
35). 


Norm and Distance in C n 


If 11 = (u\,u 9 _, u„) and v = (vj, V 2 , ..., v H ) are vectors in C n with the Euclidean inner product, then 

INI - («. «} 1/2 - /|ui| 2 4 |h 2 | 2 H-h |u H | 2 

and 


d (ll, v) = ||ll — v|| = ill — V, U — Vj, 


1/2 




|«1 ~ V 11 2 4 1^2 — v 2 | 2 H-h |W„-V„| 2 


Observe that these are just the formulas for the Euclidean norm and distance discussed in Section 10.4. 


♦ 


Norm of a Function in Complex CfO, 2n] 


Calculus Required 

If complex C[0, 2 tt] has the inner product of Example 3, and if f — s ™ x , where m is any integer, then with the help of Formula 
15 of Section 10.3, we obtain 

||f|| = (f,f} 1/2 = 


♦ 



Orthogonal Sets 
















The definitions of such terms as orthogonal vectors, orthogonal set, orthonormal set, and orthonormal basis carry over to 
complex inner product spaces without change. Moreover, Theorems 6.2.4, 6.3.1, 6.3.3, 6.3.4, 6.3.5, 6.3.6, and 6.5.1 remain 
valid in complex inner product spaces, and the Gram-Schmidt process can be used to convert an arbitrary basis for a complex 
inner product space into an orthonormal basis. 


EXAMPLE 6 

Orthogonal Vectors in q2 


The vectors 

11 = (i, 1 ) and v=(l,i) 

in q '2 are orthogonal with respect to the Euclidean inner product, since 

n-v= (0(1) + (l)CO (0(1) + (1)( -0 = 0 

♦ 

EXAMPLE 7 

Constructing an Orthonormal Basis for £3 



Consider the vector space with the Euclidean inner product. Apply the Gram-Schmidt process to transform the basis 
vectors Ul = (i, i, i), u 2 = (0, i, i), 113 = (0, 0, i) into an orthonormal basis. 


Solution 

Step 1. yj = m = (;, i, i ) 


Step 2. 


v 2 = «2 - proj^ «2 = «2 - - 2 vi 

llvill 


( 0 , i, i ) - -|(i, i, i ) = 


-H4- 


Step 3. 


V3 = u 3 - P ro Jf^2 11 


fu 3 ,vi) (U3,v 2 ) 

= u 3 - - T~ vi - ->- t 1 - v; 


llvil 


|V2l 


= (0, 


= fo, — 4-!, 4 i 


2 ’ 2 


Thus 


2 . 1 . 1 . 
V2 = l- 3 i. 3 i. 3 i 


vi = (i,0. 

form an orthogonal basis for c 3 - The norms of these vectors are 


V3= [0, 


|vi 




Ivall - 




IIV 3 II - 


& 










so an orthonormal basis for q 3 is 


XI 

vi 


II 


J_ i_ i } 



2 i i i \ 

j/?' jef 


n 

V3 


II 


o. 


i i \ 
& 


♦ 


Orthonormal Set in Complex C[0, 2 tt] 


Calculus Required 


Let complex C[0, 2 jt] have the inner product of Example 3, and let IT be the set of vectors C[0, 2 jt] in of the form 

e 1TriX = cos wix 4- i sin mx 

where m is an integer. The set W is orthogonal because if 

f = e ikx and g = e iSx 

are distinct vectors in W, then 


(f,g } = jf "e ikx e ih dx = \ ikx e~ ilx dx = \ 

/■2tF J-27T 

= / cos (k — l)xdx+ij sin (k — l)xdx 

Jo J 0 




1 


sin(£ — l)x 


2 7T 


— i 


JO 


1 


k — I 


cos (k — I)x 


2 TV 


JO 


k^I 
= (0)-i(0)=0 

If we normalize each vector in the orthogonal set W, we obtain an orthonormal set. But in Example 5 we showed that each 
vector in W has norm ^2ir, so the vectors 

m = 0 , ± 1 , ± 2 ,... 


1 imx 




form an orthonormal set in complex C[0, 2~]. 


Exercise Set 10.5 



Click here for Just Ask! 


Let u = (u j, U 2 ) ar| d v = (vi, V 2 )- Show that (u, vj = 3u\v\-\ 2u'jy2 defines an inner product on re¬ 


compute (u, v} using the inner product in Exercise 1. 
















(a) u=(2z, — i), v = ( — i, 3i) 


(b) u = (0, 0), v= (1 -i,7-5i) 

(c) n=(l + i, l-i).v=(l-i, l4i) 


(d) u — (3 i, - 1 + 2 i), v — (3 i, - 1 I 2 i) 


Let u = (u j, u 2 ) and v = (v i, v 2 ) • Show that 

^ _ _ _ _ 

(u, v) = uivi + (1 \-i)u\V 2 +( 1 =i)u 2 Vi + 3u2V2 

defines an inner product on C' 2 . 

Compute (u, v} using the inner product in Exercise 3. 

4. 

(a) u=(2:, — :)> v = ( — 3i) 

(b) u = (0, 0), v= (1 

(c) u = (14 i, 1 - i)> v = (1 - i, 1 4 i) 

(d) u = (3 i, - 1 I 2 i), v = (3 i, -1 + 2 i) 


Let u= (y 1; u 2 ) ar *d v = (vi, V 2 )- Determine which of the following are inner products on q-. For those that are not, list 
the axioms that do not hold. 


(a) (u, vJ = uivj 

(b) (u, v} = uivi-U 2 V 2 

(c) (u, v}= |u 1 | 2 |v 1 | 2 + |u 2 | 2 |v 2 | 2 

(d) (u, v} = 2u\v\ + zuiV2 4 iu 2 V\ 4 2^2 

(e) |u, vj = 2u\v\ + ZWJV2 — ^2^1 I 2 & 2 V 2 



Use the inner product of Example 2 to find ( U, V} if 


u = 

— i 1 4- i 

and 

V = 

'3 

— 2 — 3i" 


1 — i i 



4 j 

1 


Let u = (hi, u% tii) and v = (vi, V 2 , V 3 )- Does (u, vj = hjvi + H2 v 2 f U 3 V 3 - iu 3 v l define an inner product on c 3 ? If not, 
list all axioms that fail to hold. 

Let V be the vector space of complex-valued functions of the real variable x, and let { = f i (?;) -\ if 2 (jr) and 
g = gj (x) f ig 2 (x) be vectors in V. Does 

(f.g}=C/i(0)4 i/2(0))(gi(0) I ig 2 (0)) 
define an inner product on V? If not, list all axioms that fail to hold. 

Let C l have the inner product of Exercise 1. Find ||w|| if 

9. 

(a) w = ( — i, 3i) 


(b) w= 0 — 1 -FO 


(c) w =(0,2-:) 

(d) w = (0, 0) 


For each vector in Exercise 9, use the Euclidean inner product to find ||^v|| 

10 . 


Use the inner product of Exercise 3 to find ||w|| if 

11 . 

(a) w= (1, — z) 


(b) w= (1 — s, 1 -F s) 


(c) w = (3 — Ai, 0) 

(d) w = (0, 0) 


Use the inner product of Example 2 to find || j4|| 

12 . 


(a) 


A = 


— i 1 i 
6i 2 i 



(b) 


Let have the inner product of Exercise 1. Find d(\, v) if 

(a) x= (1, l),y= (i, -:) 

(b) x = (1 -i, 3 + 2), y = (1 -t i, 3) 


Repeat the directions of Exercise 13 using the Euclidean inner product on £*2. 

14. 

Repeat the directions of Exercise 13 using the inner product of Exercise 3. 

15. 

Let complex M 22 have i nner product of Example 2. Find d(A, B) if 

16. 


< a > 21 = 

i 5i 

and B = 

' -5i 

0" 



8z 3z 



li 

— 3i_ 


II 

' - 1 

1 — z 

and 

B = 

'2 i 2- 

- 3i 


1 + i 

2 



i 

1 


Let c 3 have the Euclidean inner product. For which complex values of k are u and v orthogonal? 

17. 

(a) u= (2i, i, 3z)> v= (z, 6 i, k) 

(b) u = (£, k, 1 +■:), v = (1, — 1, 1 — z) 


18 . 


Let complex M 22 h ave i nner product of Example 2. Determine which of the following are orthogonal to 

2 i i 
- i 3 i 


A = 


(a) 


-3 1 -i 
1 -i 2 


1 1 
0 - 1 


(b) 



(C) 


0 0 
0 0 


(d) 


0 1 
3-i 0 


19 Let c 3 have the Euclidean inner product. Show that for all values of the variable Q, the vector - = -4=, -4= 

W 3 P V 3 , 

norm 1 and is orthogonal to both (1,0) and (0, j, — j). 

Let have the Euclidean inner product. Which of the following form orthonormal sets? 

20 . 

(a) (i,0), (0,1-0 


(b) 


i i i i 


l l 


(c) \k p\ 1 " & “ 


(d) (j,0),(0,0) 


21 . 


Let c 3 have the Euclidean inner product. Which of the following form orthonormal sets? 


(a) 


> 0 ’ knk k 0 ’ k 


(b) 


— j _2j 14 fli lj -14 fli 1, 1, 
3 1 ’ 3 ! ’3 ! M3 ! ’3 !i 3 ! M3 ! ’3 ! ’3 ! 


( c ) f i i 2i 1 I i i 

{& j' \p’ {2 


, 0 


Let 


22 . 


x=M=, and y=' 2 * 3 * 


VS’ ' 1/30'^ 

Show that {.v, v) is an orthonormal set if ^2 has the inner product 


has 




(u, v} = 3uivi +2U2V2 

but is not orthonormal if C ' 2 has the Euclidean inner product. 

Show that 

23, m = (i, 0, 0, j), U 2 = (“i, 0, 113 = (2i, 3i, 2i, - 2i), 114= ( -i, 2i, -i, i) 

is an orthogonal set in C 4 with the Euclidean inner product. By normalizing each of these vectors, obtain an orthonormal 
set. 

Let c 2 have the Euclidean inner product. Use the Gram-Schmidt process to transform the basis (m, 112 } into an 
orthonormal basis. 


( a ) ui = (z, — 3i)> U 2 = (2 i, 2i) 


(b) ui = ( i, 0)> 112 = (3 i, — 5z) 


Let C 2 have the Euclidean inner product. Use the Gram-Schmidt process to transform the basis {u 1; u 2i 113} into an 
orthonormal basis. 


( a ) ui = ( i, i. O’ U2 = C — h h 0)> 113 = (i, 2i, z) 


(b) m = (j, 0, 0), 112 = (3z, 7 i, - 2i), 113 = (0, 4 i, i ) 


Let C 4 have the Euclidean inner product. Use the Gram-Schmidt process to transform the basis ( Ul; U 2 , 113 , 114 } into an 
orthonormal basis. 

ui = (0, 2z, z, 0), 112 = (i, — z, 0, 0), 113 = (z, 2z, 0, — j), 114= (j, 0, z, j) 

Let c 3 have the Euclidean inner product. Find an orthonormal basis for the subspace spanned by ( 0 , j, 1 — i) and 

27 • C-i. 0,1 + 0- 

Let have the Euclidean inner product. Express the vector w = ( = ^ 2i, 6i, 0) in the form w = wi 4 W 2 > where the vector 
wi is in the space W spanned by = (— j, 0,z, 2i) and 112 = (0, z, 0, 0’ anc * w 2 i s orthogonal to W. 


29. 

(a) Prove: If k is a complex number and (1 1, v) is an inner product on a complex vector space, then 

{11 — kv, u — = (u, 11 J — £(u, vj — u, vj, 4 kkfv, v\. 

(b) Use the result in part (a) to prove that 0 < fu, uj — k(\i, vj — kfn, vj 4 kklv, vj. 


Prove that if u and v are vectors in a complex inner product space, then 

I(n, v}| 2 <(u,u}(v, V} 


30 . 






This result, called the Cauchy-Schwarz inequality for complex inner product spaces , differs from its real analog 
(Theorem 6.2.1) in that an absolute value sign must be included on the left side. 

Hint Let k = (u, vj / (v, v], in the inequality of Exercise 29(b). 


Prove: If u= (m, U 2 , ..., w H ) and v = (vj, V 2 , v M ) are vectors in C n , then 

1/2 1/2 

|uivi + uyv'j I I (lull 2 ■ \u'j\“ | | |Uv,|“) (Ivi | 2 | Iv^l 2 +■■■ I |v H r) 

This is the complex version of Formula 4 in Theorem 4.1.3. 

Hint Use Exercise 30. 


Prove that equality holds in the Cauchy-Schwarz inequality for complex vector spaces if and only if u and v are linearly 
32. dependent. 


Prove that if (u, vj is an inner product on a complex vector space, then 

( 0 , vj = (v, 0 } = 0 

Prove that if (u, vj is an inner product on a complex vector space, then 
(u, v I w} = (u, vj 4 (u, w} 


Theorems 6.2.2 and 6.2.3 remain true in complex inner product spaces. In each part, prove that this is so. 

35. 


(a) Theorem 6.2.2a 

(b) Theorem 6.2.2 b 

(c) Theorem 6.2.2c 

(d) Theorem 6.2.2 d 

(e) Theorem 6.2.3 a 

(f) Theorem 6.2.3 b 

(g) Theorem 6.2.3c 

(h) Theorem 6.2.3 d 


36 . 


In Example 7 it was shown that the vectors 

i i i ) 

f3’ fr 


V2 = 


2 i i i \ 

{s’ {If 


v 3 = 



i i \ 


VI = 



form an orthonormal basis for c 3 - Use Theorem 6.3.1 to express u = (1 — z, 1 4 z, 1) as a linear combination of these 
vectors. 

Prove that if u and v are vectors in a complex inner product space, then 
(u, v} = j||u I v || 2 — ^-||u — v|| 2 4 ^||u + zvH 2 -U||u-zv || 2 

Prove: If (y 1; y 2 i v H } i s an orthonormal basis for a complex inner product space V, and if u and w are any vectors in 
38 - v, then 

(u, w} = (u, VI }(w, VI } 4 (u, V 2 }(w, v 2 } h I (u, v M }{w, t h J 
Hint Use Theorem 6.3.1 to express u and w as linear combinations of the basis vectors. 


39. (For Readers Who Have Studied Calculus) Prove that if f = f j (*) _| z/ 2 (x) and g = g^(x) f zg 2 (x) are vectors in 
complex C[a,b] then the formula 


(f, S}= / [/l0)4 j/2W][glW4 *£ 2 ( x )] 

Ja 

defines a complex inner product on C[a,b]. 

40. (For Readers Who Have Studied Calculus) Let f = x and g = 1 -f ix be vectors in complex C[0, 1 ] and let this space 
have the inner product defined in Exercise 39. Find 


(a) llsll 

(b) (f, g( 
(O (g.f> 


41. (For Readers Who Have Studied Calculus) Let f = /1 (x) 4 z/ 2 (x) and g = gi (x) 4 zg 2 (x) be vectors in complex 
CfO, 1 ] and let this space have the inner product defined in Exercise 39. Show that the vectors where m = Cu ± 1, 

j_ 2 , form an orthonormal set. 
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10.6 

UNITARY, NORMAL, 
AND HERMITIAN 
MATRICES 


For matrices with real entries, the orthogonal matrices {A -1 — A 1 ) and the 
symmetric matrices (A = A r ) played an important role in the orthogonal 
diagonalization problem (Section 7.3). For matrices with complex entries, the 
orthogonal and symmetric matrices are of little importance; they are 
superseded by two new classes of matrices, the unitary and Hermitian 
matrices, which we shall discuss in this section. 


Unitary Matrices 

If A is a matrix with complex entries, then the conjugate transpose of A, denoted by *, is defined by 

A * =A T 

where A is the matrix whose entries are the complex conjugates of the corresponding entries in A and A is the transpose of A- 
The conjugate transpose is also called the Hermitian transpose. 


Conjugate Transpose 


If 


so 



— i 0 
3 — 2 z i 


then 



i 

34 2 i 



A* =A T 


1 -i 2 
i 3 + 2 i 
0 — i 


♦ 


The following theorem shows that the basic properties of the conjugate transpose are similar to those of the transpose. The 
proofs are left as exercises. 


THEOREM 10.6.1 




e Transpose 

of Section 4.1 that if u and v are column vectors in R n , then the Euclidean inner product on 


c y?&mPWhW-\Mtii - V = vectors in C”, then the Euclidean 

inner product on C M can be expressed as u . v = u*v- 


(a) (A ) =A 

Recall that a matrix with real entries is called orthogonal if A -1 =A T - The complex analogs of the orthogonal matrices are 

called unitary matrices. They are defined as follows: 

(b) (A +B) =A +B 

























A square matrix A with complex entries is called unitary if 

a~'=a‘ 

The following theorem parallels Theorem 6.6.1. 

THEOREM 10.6.2 

Equivalent Statements 

If A is an nx n matrix with complex entries, then the following are equivalent. 

(a) A is unitary. 

(b) The row vectors ofAfarm an orthonormal set in C n with the Euclidean inner product. 

(c) The column vectors ofAfarm an orthonormal set in C n with the Euclidean inner product. 


EXAMPLE 2 A 2 x 2 Unitary Matrix 


The matrix 


A = 


1 I ; 1 I i 

2 2 

1 —; — 1 | i 

2 2 


has row vectors 

Relative to the Euclidean inner product on C”, we have 


i'2 = 


1 —; — l + i 


2 ’ 


and 


Fill = 


lira II = 


l’l -1’2 - 


1 I i 


+ 


1 I i 


1 — i 


+ 


— l+i 


" 1^2 ' 2 _1 

= {l h 2 = 


111 


l+i. ^ / l+i 


^)H 


— l —; 

2 


= = 0 
2 2 

so the row vectors form an orthonormal set in cfa. Thus A is unitary and 


(1) 










































( 2 ) 


= A 


1 — i 1 I i 

2 2 

1 — i — 1 — i 

2 2 


The reader should verify that matrix 2 is the inverse of matrix 1 by showing that AA * = A* A = l- 


♦ 



Charles Hermite (1822-1901) was a French mathematician who made fundamental contributions to algebra, matrix theory, 
and various branches of analysis. He is noted for using integrals to solve a general fifth-degree polynomial equation. He also 
proved that the number s (the base for natural logarithms) is a transcendental number—that is, a number that is not the root 
of any polynomial equation with rational coefficients. 


Recall that a square matrix A with real entries is called orthogonally diagonalizable if there is an orthogonal matrix p such that 
P~^AP( = P t AP ) is diagonal. For complex matrices we have an analogous concept. 


DEFINITION 


A square matrix A with complex entries is called unitarily diagonalizable if there is a unitary P such that p 1 A C '( = P * AP) 
is diagonal; the matrix P is said to unitarily diagonalize A- 


We have two questions to consider: 

Which matrices are unitarily diagonalizable? 

How do we find a unitary matrix p to carry out the diagonalization? 


Before pursuing these questions, we note that our earlier definitions of the terms eigenvector, eigenvalue, eigenspace, 
characteristic equation, and characteristic polynomial carry over without change to complex vector spaces. 


Hermitian Matrices 













In Section 7.3 we saw that the problem of orthogonally diagonalizing a matrix with real entries led to consideration of the 
symmetric matrices. The most natural complex analogs of the real symmetric matrices are the Hermitian matrices, which are 
defined as follows: 


DEFINITION 


A square matrix A with complex entries is called Hermitian if 

A = A 


EXAMPLE 3 A 3 x 3 Hermitian Matrix 


If 


then 


A = 










1 i 

1 + i 





A = 

-i -5 

2-i 





1 — i 2 -|- i 

3 



1 — i 

1 — i 



1 

i 1 4- i 

i -5 

2 4- i 

a * 

, so A = 

a t = 

— i 

-5 2-i 

1 + i 2—i 

3 



1 — i 

2 + i 3 


= A 


which means that A is Hermitian. 


It is easy to recognize Hermitian matrices by inspection: As seen in 3, the entries on the main diagonal are real numbers, and the 
“mirror image” of each entry across the main diagonal is its complex conjugate. 


I t W 
) - / 2 A-i 3 


( 3 ) 


Normal Matrices 

Hermitian matrices enjoy many but not all of the properties of real symmetric matrices. For example, just as the real symmetric 
matrices are orthogonally diagonalizable, so we shall see that the Hermitian matrices are unitarily diagonalizable. However, 
whereas the real symmetric matrices are the only matrices with real entries that are orthogonally diagonalizable (Theorem 7.3.1), 
the Hermitian matrices do not constitute the entire class of unitarily diagonalizable matrices; that is, there are unitarily 
diagonalizable matrices that are not Hermitian. To explain why this is so, we shall need the following definition: 


DEFINITION 


A square matrix A with complex entries is called normal if 


AA =A A 






















Hermitian and Unitary Matrices 


Every Hermitian matrix A is normal since AA * = AA = a * a, and every unitary matrix A is normal since AA * = / = A * A 


♦ 


The following two theorems are the complex analogs of Theorems 7.3.1 and 7.3.2. The proofs will be omitted. 


THEOREM 10.6.3 

Equivalent Statements 

If A is a square matrix with complex entries, then the following are equivalent: 

(a) A is unitarily diagonalizable. 

(b) A has an orthonormal set of n eigenvectors. 

(c) A is normal. 


THEOREM 10.6.4 


If A is a normal matrix, then eigenvectors from different eigenspaces of A are orthogonal. 


Theorem 10.6.3 tells us that a square matrix A with complex entries is unitarily diagonalizable if and only if it is normal. 
Theorem 10.6.4 will be the key to constructing a matrix that unitarily diagonalizes a normal matrix. 

Diagonalization Procedure 

We saw in Section 7.3 that a symmetric matrix A is orthogonally diagonalized by any orthogonal matrix whose column vectors 
are eigenvectors of A- Similarly, a normal matrix A is diagonalized by any unitary matrix whose column vectors are eigenvectors 
of A- The procedure for diagonalizing a normal matrix is as follows: 


Step 1. Find a basis for each eigenspace of A- 

Step 2. Apply the Gram-Schmidt process to each of these bases to obtain an orthonormal basis for each eigenspace. 

Step 3. Form the matrix P whose columns are the basis vectors constructed in Step 2. This matrix unitarily diagonalizes A- 

The justification of this procedure should be clear. Theorem 10.6.4 ensures that eigenvectors from different eigenspaces are 
orthogonal, and the application of the Gram-Schmidt process ensures that the eigenvectors within the same eigenspace are 
orthonormal. Thus the entire set of eigenvectors obtained by this procedure is orthonormal. Theorem 10.6.3 ensures that this 
orthonormal set of eigenvectors is a basis. 






Unitary Diagonalization 


The matrix 



is unitarily diagonalizable because it is Hermitian and therefore normal. Find a matrix p that unitarily diagonalizes p. 


Solution 


The characteristic polynomial of A is 

det(A7 — A) — det 

so the characteristic equation is 


A — 2 -1- 

-1 f i A — 3 


= (A — 2) (A — 3) — 2 = A 2 


A 2 — 5A -|- 4 = (A — 1) (A — 4) = 0 


and the eigenvalues are A = 1 and A = 4- 


5A I 4 


By definition, 


x = 


*1 

*2 


will be an eigenvector of A corresponding to A if and only if x is a nontrivial solution of 


A— 2 -1-f 

*1 


'O' 

_-l+i A — 3_ 

_*2_ 


_ 0 _ 


(4) 


To find the eigenvectors corresponding to A = 1, we substitute this value in 4: 


- 1 

— 1 — i 

*1 


'o' 

_ — 1 A i 

-2 

_* 2 _ 


_0_ 


Solving this system by Gauss-Jordan elimination yields (verify) 

*l = (-l-zK *2 = s 

Thus the eigenvectors of A corresponding to A = 1 are the nonzero vectors in c 2 of the form 


v — 

■(-l-i)/ 

— C 1 

— 1 — i 

A - 

s 

— o 

1 


Thus this eigenspace is one-dimensional with basis 

— 1 — i 


In this case the Gram-Schmidt process involves only one step: normalizing this vector. Since 

||u|| = /|-l-j| 2 d |1| 2 = ^2+ l = /3 

the vector 


Pi = 



— 1 —; 

(3 

1 

P 


( 5 ) 


































is an orthonormal basis for the eigenspace corresponding to A = 1. 


To find the eigenvectors corresponding to A — 4, we substitute this value in 4: 


2 

— 1 — i 

*1 


'o' 

_ — 1 + i 

1 

_* 2 _ 


_0_ 


Solving this system by Gauss-Jordan elimination yields (verify) 

11 = 


X2 = s 


so the eigenvectors of A corresponding to A — 4 are the nonzero vectors in fj 1 of the form 





1 + i 

X = 

{ 2 I s 

= s 

2 


s 


1 


Thus the eigenspace is one-dimensional with basis 


u = 


1 + i 

2 

1 


Applying the Gram-Schmidt process (that is, normalizing this vector) yields 

1 + i 


P2 = 


U 


U 


Thus 


P= [pilP2] = 


fe 

_ 2 _ 


— 1 — i 1 + ; 

ft 

1 2 

ft ft 


diagonalizes A and 


P~ X AP = 


1 0 
0 4 


Eigenvalues of Hermitian and Symmetric Matrices 

In Theorem 7.3.2 it was stated that the eigenvalues of a symmetric matrix with real entries are real numbers. This important 
result is a corollary of the following more general theorem. 


THEOREM 10.6.5 


The eigenvalues of a Hermitian matrix are real numbers. 


Proof If A is an eigenvalue and v a corresponding eigenvector of an nxn Hermitian matrix ,4, then 

j4v = Av 

If we multiply each side of this equation on the left by v * and then use the remark following 






























Theorem 10.6.1 to write v*v=||v|| 2 (with the Euclidean inner product on C H ), then we obtain 

v Alv = v (Av) = Av v = A||v|| 2 

But if we agree not to distinguish between the 1 x 1 matrix v *Av and its entry, and if we use the fact 
that eigenvectors are nonzero, then we can express \ as 


Thus, to show that \ is a real number, it suffices to show that the entry of v *Av is real. One way to do 
this is to show that the matrix v *Av is Hermitian, since we know that Hermitian matrices have real 
numbers on the main diagonal. However, 

(v Ah') = v A (v ) = v Alv 

which shows that v *Av is Hermitian and completes the proof. 


The proof of the following theorem is an immediate consequence of Theorem 10.6.5 and is left as an exercise. 


THEOREM 10.6.6 


The eigenvalues of a symmetric matrix with real entries are real numbers. 


Exercise Set 10.6 



Click here for Just Ask! 


In each part, find a * • 

1 . 


(a) 2 i 1 — i 

A= 4 3 -hi 

5 4-i 0 


(b) 2i 1 — i 
A= 4 5-7 i 
i 3 


— i 4- 1 
— i 
1 


( c ) A= [7i 0 — 3i] 


(d) 


A = 


a 11 «12 «13 
«21 «22 «23 












Which of the following are Hermitian matrices? 


2. 


(a) 


0 i 
i 2 


(b) 


1 14 i 

1 -i -3 


(c) 


i 

i 


(d) 


— 2 1 — i — 1 4 i 

\+i 0 3 

— 1 — i 3 5 


(e) 


1 0 0 
0 1 0 
0 0 1 


3. 


Find l, and m to make A a Hermitian matrix. 


A = 


— 1 k —i 
3 — 5i 0 m 
l 2 I 4j 2 


4. 


Use Theorem 10.6.2 to determine which of the following are unitary matrices. 


5. 


(a) 

In each 




i 0 

|p@rt,jv|erify that the matrix is unitary and find its inverse. 


1 

5 5 


(b) 

(c) 


i 1, 

5 5 




1 1 

{l 

1 I ; 1 I i 

2 2 


, 1 


^ +i) 
{2 {e 




1 I i 
2 


(d) 


3 I i 
2/15" 



1 

/3 

4-t 3i 
2/l5 


1 

2 


£ 


5i 

2/l5 


Show that the matrix 


6 . 


1 


JO 


—iO 


JO 


is 


— is 


—iO 


is unitary for every real value of 0. 

In Exercises 7-12 find a unitary matrix p that diagonalizes A, and determine p~^AP- 


7. 


A = 


4 

1 4 i 


1 —; 
5 



3 


i 


9. 


A = 


6 

2-2 i 


2 4 2j 
4 


10 . 


0 34 i 

3 — i -3 


11 . ,4 = 


0 

- 1 

— 1 — i 


0 

— 1 f j 
0 


12 . 


A = 



f2 

0 

2 


Show that the eigenvalues of the symmetric matrix 

A = \ 1 4i 
|_4; 3 _ 

are not real. Does this violate Theorem 10.6.6? 


14 . 


(a) Find a 2 x 2 matrix that is both Hermitian and unitary and whose entries are not all real numbers. 



(b) What can you say about the inverse of a matrix that is both Hermitian and unitary? 


Prove: If A is an n x « matrix with complex entries, then det(A') = det(A'). 

15. 

Hint First show that the signed elementary products from A are the conjugates of the signed elementary products from A- 


16. # _ 

(a) Use the result of Exercise 15 to prove that if A is an n x n matrix with complex entries, then det(j4 ) = det(j4). 


(b) Prove: If A is Hermitian, then det(j4) is real. 

(c) Prove: If det(j4) is unitary, then |det(j4) | = 1. 


Prove that the entries on the main diagonal of a Hermitian matrix are real numbers. 

17. 


Let 



'<* 11 

<*12 

<* 13 _ 


*11 

*12 

*13 

A = 

<*21 

<*22 

<*23 

and B = 

*21 

*22 

*23 


<*31 

<*32 

<*33 


*31 

*32 

LO 

LO 


be matrices with complex entries. Show that 

(a) (.4*)* =,4 

(b) (A \-B)*=A* P5* 

(c) (kA) =kA 

(d) (AB) —B A 


^ Prove: If A is invertible, then so is a *, in which case (j4 + ) 1 = (j4 -1 )*. 

Show that if A is a unitary matrix, then A * is also unitary. 

20 . 

Prove that an n x n matrix with complex entries is unitary if and only if its rows form an orthonormal set in C M with the 
21- Euclidean inner product. 




Use Exercises 20 and 21 to show that an « x n matrix is unitary if and only if its columns form an orthonormal set in Q n 

22. with the Euclidean inner product. 

Let A and ,. be distinct eigenvalues of a Hermitian matrix A- 

23. 

(a) Prove that if x is an eigenvector corresponding to \ and y an eigenvector corresponding to it , then x *yly = Ax*y and 

* * 
x Ay = fix y. 

(b) Prove Theorem 10.6.4. 
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Chapter 10 


Supplementary Exercises 

Let u — (it i, u %and v = (v \, V 2 , v H ) be vectors in C M , and let n = (aj, H 2 , u H ) and v = (vi, i?2> v H )- 
(a) Prove: vFv = u - v. 


(b) Prove: u and v are orthogonal if and only if u and v are orthogonal. 


Show that if the matrix 
a b 
— b a 

is nonzero, then it is invertible. 

Find a basis for the solution space of the system 


' -1 

— i 1 



"O' 

— i 

1 i 

*2 

= 

0 

1 

i — 1 

x 3 


0 


4. 


Prove: If a and b are complex numbers such that |a| 2 + |&| 2 = 1, and if 0 is a real number, then 


A = 


a b 

iO~r iO- 

— e b e a 


is a unitary matrix. 

Find the eigenvalues of the matrix 
'0 0 1 
1 0 Lij + 1 H 1 

o 1 


6 . 


where w — s 2^/3 

Consider the relation between the complex number z = a-\-ib and the corresponding 2x2 matrix with real entries 

7 _\ a 

b a 


(a) How are the eigenvalues of Z related to 2 ? 


















(b) How is the complex number z related to the determinant of Z? 


(c) Show that Z _1 corresponds to z _1 • 


(d) Show that the product (a + bi) (c 4 - di) corresponds to the matrix that is the product of the matrices corresponding 
to a | bi and c 4 di- 


7. 


(a) Prove that if 2 is a complex number other than 1, then 


1 +z4-z 2 4-- + z m = 


1 — z‘ 


hH-1 


1 —z 


Hint Let 5 be the sum on the left side of the equation and consider the quantity s-zS- 

(b) Use the result in part (a) to prove that if s-” = 1 and z* b then 1 4 z \ z 2 \ ..._| — 0 . 


(c) Use the result in part (a) to obtain Lagrange's trigonometric identity 

1 sin I" 0 f ^)9 

14-cos# I cos 29 4-h cos nB = I — L . 2 J 

2 2sm(y/2) 

for 0 < 9 < 2". 

Hint Let z=cos#H isinii- 

Let uj = g 2 ^ 73 . Show that the vectors V i = (1 / ^3) (1, 1, 1)- v 2 = (1 f |/3) (1, w, u?) and v 3 = (1 / ^3) (1, ^ 2 , u 4 ) form 
an orthonormal set in C' 3 . 

Hint Use part (b) of Exercise 7. 


9. 


Show that if jj is an n x « unitary matrix and |zi | = |z 2 | = — = |z H | = 1 , then the product 


z 1 0 0 



0 

0 


0 0 0 z n 


is also unitary. 


Suppose that A* = — A- 

10 . 


(a) Show that iA is Hermitian. 


(b) Show that A is unitarily diagonalizable and has pure imaginary eigenvalues. 



Show that the eigenvalues of a unitary matrix have modulus 1. 

11 . 

Under what conditions is the following matrix normal? 

a 0 0 
0 0c 
0 b 0 

Show that if u is a nonzero vector in C n that is expressed in column form, then P = uu * is Hermitian and has rank 1. 

13. 

Show that if u is a unit vector in C n that is expressed in column form, then H — 1 — 2uu* is unitary and Hermitian. This 
14* is called a Householder matrix. 

What geometric interpretations might you reasonably give to multiplication by the matrices p = uu * and H = 1 — 2uu * 
in Exercises 13 and 14? 
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Chapter 10 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Sections 10.1 and 10.2 

Tl. (Complex Numbers and Numerical Operations) Read your documentation on entering and displaying complex numbers 
and for performing the basic arithmetic operations of addition, subtraction, multiplication, and division. Experiment with 
numbers of your own choosing until you feel you have mastered the operations. 


T2. (Matrices with Complex Entries) For most technology utilities the procedures for adding, subtracting, multiplying, and 
inverting matrices with complex entries are the same as for matrices with real entries. Experiment with these operations on 
some matrices of your own choosing, and then try using your utility to solve some of the exercises in Sections 10.1 and 10.2. 


T3. (Complex Conjugate) Read your documentation on finding the conjugate of a complex number, and then use your utility to 
perform the computations in Example 1 of Section 10.2. 


Section 10.3 

Tl. (Modulus and Argument) Read your documentation on finding the modulus and argument of a complex number, and then 
use your utility to perform the computations in Example 1. 


Section 10.6 

Tl. (Conjugate Transpose) Read your documentation on finding the conjugate transpose of a matrix with complex entries, and 
then use your utility to perform the computations in Examples Example 1 and Example 3. 


T2. (Unitary Diagonalization) Use your technology utility to diagonalize the matrix A in Example 5 and to find a matrix p that 
unitarily diagonalizes A- (See Technology Exercise Tl of Section 7.2.) 
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CHAPTER 


Applications of Linear Algebra 


NTRODUCTION This chapter consists of 21 applications of linear algebra. With one clearly marked exception, each 
application is in its own independent section, so sections can be deleted or permuted as desired. Each topic begins with a list 
of linear algebra prerequisites. 

Because our primary objective in this chapter is to present applications of linear algebra, proofs are often omitted. Whenever 
results from other fields are needed, they are stated precisely, with motivation where possible, but usually without proof. 
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11.1 

CONSTRUCTING CURVES 

In this section we describe a technique that uses determinants to construct 
lines, circles, and general conic sections through specified points in the plane. 

AND SURFACES 

The procedure is also used to pass planes and spheres in 3-space through 

THROUGH SPECIFIED 

fixed points. 

POINTS 

Linear Systems 


Determinants 


Analytic Geometry 

The following theorem follows from Theorem 2.3.6. 

THEOREM 11.1.1 


A homogeneous linear system with as 
the coefficient matrix is zero. 

many equations as unknowns has a nontrivial solution if and only if the determinant of 


We shall now show how this result can be used to determine equations of various curves and surfaces through specified points. 


A Line through Two Points 

Suppose that (*^ y^') and 72 ) are two distinct points in the plane. There exists a unique line 

ci* 4- c-yy V C2 = 0 


that passes through these two points (Figure 11.1.1). Note that c\, C2, and C3 are not all zero and that these coefficients are unique 
only up to a multiplicative constant. Because (* y^) and (^ 2 , 72 ) on the line, substituting them in 1 gives the two equations 


cm + coy 1 + c? = 0 


( 2 ) 


C 1*2 + cyyi + ^3 = 0 


(3) 



Figure 11.1.1 

The three equations, 1, 2, and 3, can be grouped together and rewritten as 






XC\ + yc 2 h C 3 = 0 
*1^1 +71^2 h c 3 = 0 
X2 C 1 4 y2C7 h c 3 = 0 

which is a homogeneous linear system of three equations for cj, C 2 , and c 3 . Because cj, C 2 , and C 3 are not all zero, this system has 
a nontrivial solution, so the determinant of the system must be zero. That is, 

x y 1 

*1 y 1 1 

*2 yi 1 

Consequently, every point ( x> y) on the line satisfies 4; conversely, it can be shown that every point ( x> y) that satisfies 4 lies on 
the line. 


= 0 


(4) 


Equation of a Line 


Find the equation of the line that passes through the two points (2, 1) and (3,7). 


Solution 

Substituting the coordinates of the two points into Equation 4 gives 

x y 1 


2 1 1 
3 7 1 


= 0 


The cofactor expansion of this determinant along the first row then gives 

— 6x y +11 = 0 


A Circle through Three Points 

Suppose that there are three distinct points in the plane, (*^ y^), (% 2 , 72 )’ anc * (* 3 , 73 )’ not lying on a straight line. From 
analytic geometry we know that there is a unique circle, say, 

c 1 (x 2 4 y 2 ) 4 c 2 x 4 c-yy 4 c 4 = 0 ( 5 ) 

that passes through them (Figure 11.1.2). Substituting the coordinates of the three points into this equation gives 

Cl(*i +y?) 4 C 2*1 + C 3 l*T + C 4 = 0 ( 6 ) 


^1 (*2 4 yh + C 2 X 2 + c 3 y 2 + c 4 = 0 


(V) 


ci (*3 4 y 2 ) 4 c 2 x 3 4 ^^3 + c 4 = 0 


(8) 








Figure 11.1.2 

As before, Equations 5 through 8 form a homogeneous linear system with a nontrivial solution for cj, C 2 , cy and C 4 . Thus the 
determinant of the coefficient matrix is zero: 


X 2 - 

iy 2 

X 

y 

1 

4 

1 y 2 \ 

*1 

y 1 

1 

4 

1 y\ 

x 2 

yi 

1 

4 

1 y\ 

x 3 

y3 

1 


(9) 


This is a determinant form for the equation of the circle. 


Equation of a Circle 


Find the equation of the circle that passes through the three points (1,7), (6, 2), and (4, 6). 


Solution 


Substituting the coordinates of the three points into Equation 9 gives 


which reduces to 


In standard form this is 


10(x 2 


2 2 

X 

y l 

50 

1 

7 1 

40 

6 

2 1 

52 

4 

6 1 

7 2 )- 

20x 

-407 


= 0 


-200 = 0 


(*-l) 2 + 0-2) 2 = 5 2 


Thus the circle has center (1,2) and radius 5. 


A General Conic Section through Five Points 


♦ 


The general equation of a conic section in the plane (a parabola, hyperbola, or ellipse, or degenerate forms of these curves) is given 
by ~ 

2 2 

c]X + cjxy + c-\y 4 - C 4 X + c\y + = 0 

This equation contains six coefficients, but we can reduce the number to five if we divide through by any one of them that is not 
zero. Thus only five coefficients must be determined, so five distinct points in the plane are sufficient to determine the equation of 
the conic section (Figure 11.1.3). As before, the equation can be put in determinant form (see Exercise 7): 










X 

X 

X 

X 

X 

X 


2 


2 

1 

2 

2 

2 

3 

2 

4 

2 

5 


xy 

y 2 

X 

y 

1 

*iyi 

yf 

*1 

y i 

1 

x iyi 

y\ 

*2 

72 

1 

*373 

y\ 

*3 

73 

1 

*4y 4 

y'l 

*4 

74 

1 

*575 

y\ 

*5 

75 

1 



(10) 


Equation of an Orbit 


An astronomer who wants to determine the orbit of an asteroid about the sun sets up a Cartesian coordinate system in the plane of 
the orbit with the sun at the origin. Astronomical units of measurement are used along the axes (1 astronomical unit = mean 
distance of earth to sun = 93 million miles). By Kepler's first law, the orbit must be an ellipse, so the astronomer makes five 
observations of the asteroid at five different times and finds five points along the orbit to be 

(8.025, 8.310), (10.170, 6.355), (11.202, 3.212), (10.736, 0.375), (9.092, - 2.267) 

Find the equation of the orbit. 


Solution 


Substituting the coordinates of the five given points into 10 gives 


* 2 

*7 

7 2 

* 

7 

1 

64.401 

66.688 

69.056 

8.025 

8.310 

1 

103.429 

64.630 

40.386 

10.170 

6.355 

1 

125.485 

35.981 

10.317 

11.202 

3.212 

1 

115.262 

4.026 

0.141 

10.736 

0.375 

1 

82.664 

-20.612 

5.139 

9.092 

-2.267 

1 


The cofactor expansion of this determinant along the first row is 

386.799x 2 - 102.896xy + 446.026.y 2 - 2476.409* - 1427.971.y - 17109.378 = 0 
Figure 11.1.4 is an accurate diagram of the orbit, together with the five given points. 







Ai25, 8.3IIH 






.CEO-170, ft. 355» 












i\ 1.202, 3.212) 


.Sun 

. r\. — _ 


Ji Ui m* (U75) 











■' <y.W 1. -2.2671 


-fi -4 J1 O ? 4 fi H in l? 14 IA fX >2 










Figure 11.1.4 


A Plane through Three Points 


In Exercise 8 we ask the reader to show the following: The plane in 3-space with equation 

C\X 4- Cjy + erg + C4 = 0 

that passes through three noncollinear points (x\, y\, z{'y (x 2 , y 2 , z 2)^ and (xj, yj, 23 ) i s given by the determinant equation 

x y z 1 
*1 71 z \ 1 
x 2 72 z 2 1 
*3 73 z 3 1 


= 0 


(ID 


EXAMP Equation of a Plane 


The equation of the plane that passes through the three noncollinear points (1, 1, 0), (2, 0, -1), and (2, 9, 2) is 


X 

7 

z 

1 

1 

1 

0 

1 

2 

0 

- 1 

1 

2 

9 

2 

1 


= 0 


which reduces to 


2x — y 4- 3z — 1 = 0 


A Sphere through Four Points 

In Exercise 9 we ask the reader to show the following: The sphere in 3-space with equation 

ei (x 2 -\-y 2 + z 2 ) +C 2 X + c\y +c^z + c^ = 0 

that passes through four noncoplanar points (jq, yu Z] ), (x 2 , y% z 2)' (*3, 73. z 3)’ and (*4, 74, 2 4 ) is § iven b y the following 
determinant equation: 


2 

1 2 

1 2 




1 

X 

+ 7 

+ z 

X 

7 

z 

*1 

+ 7? 

+ 4 

*1 

71 

21 

1 

*2 

+ 72 

+4 

*2 

72 

z 2 

1 

4 

+ 73 

+4 

*3 

73 

z 2 

1 

xj 

+ 74 

+4 

*4 

74 

z 4 

1 


= 0 


( 12 ) 


EXAMPLE 5 Equation of a Sphere 


The equation of the sphere that passes through the four points (0, 3, 2), (1, -1, 1), (2, 1, 0), and (5, 1, 3) is 














This reduces to 


2 2 2 
r +/4z 


13 0 
3 1 
5 2 
35 5 


y z 1 

3 2 1 

-1 1 1 
1 0 1 
1 3 1 


= 0 


r? + y 2 + z 2 - Ax - 2y - 6z 1-5 = 0 


which in standard form is 

(*-2) 2 + O' —1) 2 + (z—3) 2 = 9 

♦ 


Exercise Set ll.l 


O 


Click here for Just Ask! 


1. 


Find the equations of the lines that pass through the following points: 


(a) (1,-1), (2, 2) 


(b) (0, 1), (1, -1) 


Find the equations of the circles that pass through the following points: 

2 . 

(a) (2, 6), (2, 0), (5, 3) 

(b) (2, -2), (3, 5), (-4, 6) 


Find the equation of the conic section that passes through the points (0, 0), (0, -1), (2, 0), (2, -5), and (4, -1). 

3. 

Find the equations of the planes in 3-space that pass through the following points: 

4. 

(a) (1, 1,-3), (1,-1, 1), (0,-1, 2) 

(b) (2,3, 1), (2,-1,-1), (1,2, 1) 


5 . 


(a) Alter Equation 11 so that it determines the plane that passes through the origin and is parallel to the plane that passes 
through three specified noncollinear points. 







(b) Find the two planes described in part (a) corresponding to the triplets of points in Exercises 4(a) and 4(b). 


Find the equations of the spheres in 3-space that pass through the following points: 

6 . 

(a) (1,2, 3), (-1,2, 1), (1, 0, 1), (1, 2, -1) 

(b) (0, 1,-2), (1,3, 1), (2,-1,0), (3, 1,-1) 


Show that Equation 10 is the equation of the conic section that passes through five given distinct points in the plane. 

7. 


Show that Equation 11 is the equation of the plane in 3-space that passes through three given noncollinear points. 

8 . 


Show that Equation 12 is the equation of the sphere in 3-space that passes through four given noncoplanar points. 

9. 


Find a determinant equation for the parabola of the form 

10. 2 

ciy + C 2 * + c-$x. bc 4 = 0 

that passes through three given noncollinear points in the plane. 

What does Equation 9 become if the three distinct points are collinear? 

11 . 

What does Equation 11 become if the three distinct points are collinear? 

12 . 

What does Equation 12 become if the four points are coplanar? 

13. 


Section 11.1 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab , Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


The general equation of a quadric surface is given by 

2 2 2 

a\x 4- ct^y 4- ct-^z 4- a$xy 4- ayxz 4- a&yz 4- a-jx 4- a%y 4- agz 4- aig = 0 
Given nine points on this surface, it may be possible to determine its equation. 



(a) Show that if the nine points y^) for j = 1, 2, 3, ..9 lie on this surface, and if they determine uniquely the 
equation of this surface, then its equation can be written in determinant form as 




z £ 

*7 

KZ 

y z 


7 

z 

1 

4 

4 

4 

*171 

* 1*1 

71*1 

*1 

71 

*1 

1 

4 

y\ 

z 2 

*272 

* 2*2 

72*2 

*2 

72 

*2 

1 

*3 

4 

z 3 

*373 

*3*3 

73*3 

*3 

73 

*3 

1 

4 

y\ 

4 

*4X4 

X 4 Z 4 

74*4 

*4 

74 

*4 

1 

4 

y 2 5 

4 

*575 

*5*5 

75*5 

*5 

75 

*5 

1 

4 

yl 

4 

*676 

* 6*6 

76*6 

*6 

76 

*6 

1 

4 

y 2 

4 

*777 

■—i 

77*7 

*7 

77 

*7 

1 

4 

yl 

4 

*878 

* 8*8 

78*8 

*8 

78 

*8 

1 

4 

yl 

4 

*979 

xgzg 

79*9 

*9 

79 

*9 

1 


(b) Use the result in part (a) to determine the equation of the quadric surface that passes through the points (1,2, 3), (2, 1, 
7), (0, 4, 6), (3, -1, 4), (3, 0, 11), (-1, 5, 8),(9, -8, 3), (4, 5, 3), and (-2, 6, 10). 


(c) Use the methods of Section 9.7 to identify the resulting surface in part (b). 


(a) A hyperplane in the ^-dimensional Euclidean space R n has an equation of the form 


a 1*1 H ^2*2 4" a 2 x 3 I -1 <3 m *h I a n+\ = 0 

where a u i = l, 2, 3, ..., n +1 , are constants, not all zero, and x u i = l, 2, 3, ..., n, are variable 
for which 

(*1,*2.*3.—> *h) 


A point 


(* 10 , *20. *30.—. *»o) <= 


lies on this hyperplane if 

fll^io+ fl2*20+«3*30 H-1 «m*m 0 f<3 H -|-l=0 

Given that the n points (x l3 , x 2 i, * 3 ,-, i= 1 / 2, 3, ..., n, lie on this hyperplane and that th 

uniquely determine the equation of the hyperplane, showthat the equation of the hyperplan 
can be written in determinant form as 


*1 

*2 

*3 ' 


i 

*11 

*21 

*31 ' 


i 

*12 

*22 

*32 • 


i 

*13 

*23 

*33 • 

J K n3 

i 

*1h 

*2h 

*3h ‘ 

J ^nn 

i 


(b) Deter mi ne the equation of the hyperplane in ^9 that goes through the following nine points: 












(1.2. 3, 4,: 

5, 6,1, 

8, 9) 

(2, 

3, 

4, 5, 

6,7, 8,5 

U) 

(3,4, 

5, ( 

5, 7, 8,5 

>. 1, 

2) 

(4, 5, 6,7, S 

3. 9, 1, 

2, 3) 

(5, 

6, 

1, 8, 

9, 1, 2,: 

3,4) 

(6,7, 

8,5 

>, 1, 2,: 

3, 4, 

5) 

(7, 8. 9, 1,: 

2, 3,4, 

5,6) 

(3, 

9, 

1, 2, 

3,4, 5, < 

3,7) 

(9,1, 

2,: 

5, 4, 5, < 

3, 7, 

8) 
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In this section basic laws of electrical circuits are discussed, and it is shown how 
these laws can be used to obtain systems of linear equations whose solutions 
ELECTRICAL N ETWORKS yield the currents flowing in an electrical circuit. 


11.2 


Linear Systems 

The simplest electrical circuits consist of two basic components: 

electrical sources denoted by -^-j 


resistors 


denoted by 


-WA- 


Electrical sources, such as batteries, create currents in an electrical circuit. Resistors, such as lightbulbs, limit the magnitudes of the 
currents. 

There are three basic quantities associated with electrical circuits: electrical potential (E), resistance (R), and current (I). These 
are commonly measured in the following units: 


E 

in volts 

(V) 

R 

in ohms 

(«) 

I 

in amperes 

(A) 


Electrical potential is associated with two points in an electrical circuit and is measured in practice by connecting those points to a 
device called a voltmeter. For example, a commonAA battery is rated at 1.5 volts, which means that this is the electrical potential 
across its positive and negative terminals (Figure 11.2.1). 


Valimcicr 



Figure 11.2.1 

In an electrical circuit the electrical potential between two points is called the voltage drop between these points. As we shall see, 
currents and voltage drops can be either positive or negative. 

The flow of current in an electrical circuit is governed by three basic principles: 

Ohm's Law. The voltage drop across a resistor is the product of the current passing through it and its resistance; that is, 

E = !R- 


2 . Kirchhoff's Current Law. The sum of the currents flowing into any point equals the sum of the currents flowing out from 
the point. 


3 Kirchhoff's Voltage Law. Around any closed loop, the algebraic sum of the voltage drops is zero. 


















Finding Currents in a Circuit 


Find the unknown currents / j, / 2 , and / 3 in the circuit shown in Figure 11.2.2. 



Figure 11.2.2 


Solution 

The flow directions for the currents / j, / 2 , and / 3 (marked by the arrowheads) were picked arbitrarily. Any of these currents that 
turn out to be negative actually flow opposite to the direction selected. 

Applying Kirchhoff's current law to points A and B yields 

I 1 —I 2 + I 3 (Pointed) 

■^3 +^2 = ^1 (Point 5) 

Since these equations both simplify to the same linear equation 

/ l -/ 2 — 1 3 = 0 


we still need two more equations to determine /j, / 2 , and uniquely. We will obtain them using Kirchhoff's voltage law. 

To apply Kirchhoff's voltage law to a loop, select a positive direction around the loop (say clockwise) and make the following 
sign conventions: 


A current passing through a resistor produces a positive voltage drop if it flows in the positive direction of the loop and a 
negative voltage drop if it flows in the negative direction of the loop. 


A current passing through an electrical source produces a positive voltage drop if the positive direction of the loop is from + 
to - and a negative voltage drop if the positive direction of the loop is from - to +. 

Applying Kirchhoff s voltage law and Ohm's law to loop 1 in Figure 11.2.2 yields 

7/i f 3/3 -30 = 0 

and applying them to loop 2 yields 


11/ 2 -3/3-50 = 0 


( 3 ) 















Combining 1, 2, and 3 yields the linear system 


h - h-h= 0 

7/1 4 3 1 3 = 30 

11/2-3/3 = 50 

Solving this linear system yields the following values for the currents: 

/l = J m (A) ’ / 2 = ^(A), i 3 =-j^(A) 

Note that / 3 is negative, which means that this current flows opposite to the direction indicated in Figure 11.2.2. Also note that we 
could have applied Kirchhoff's voltage law to the outer loop of the circuit. However, this produces a redundant equation (try it). 


Exercise Set 11.2 



Click here for Just Ask! 


In Exercises 1-4 find the currents in the circuits. 


—WA-— 


\M1 

» -WV\-II 



JV 


2 . 


S V 




4 . 































20 Si 

m 


in v 
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-W\A 1 
20 !'2 


5. 


Show that if the current in the circuit of the accompanying figure is zero, then r 4 — R 2 R 2 i Ry 



Remark This circuit, called a Wheatstone bridge circuit, is used for the precise measurement of resistance. Here, r 4 is an 
unknown resistance and r 2 , and r 2 are adjustable calibrated resistors. r 2 represents a galvanometer—a device for 
measuring current. After the operator varies the resistances R 2 , and R 2 until the galvanometer reading is zero, the formula 
j ?4 = R 2 R 2 / R\ determines the unknown resistance R^. 


D _ _ i _ 

6. Show that if the two currents labeled I in the circuits of the accompanying figure are equal, then ] 1 

*7 + * 


/ / 



Figure Ex-6 


Section 11.2 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 
































The accompanying figure shows a sequence of different circuits. 


Tl. 


(a) Solve for the current / j, for the circuit in part (a) of the figure. 


(b) Solve for the currents /j through / 3 , for the circuit in part (b) of the figure. 


(c) Solve for the currents /j through i 5 , for the circuit in part (c) of the figure. 


(d) Continue this process until you discover a pattern in the values of l v / 2 , / 3 , .... 


(e) Investigate the sequence of values for /j in each of the circuits in parts (a), (b), (c), and so on, and numerically show 
that the limit of this sequence approaches the value 
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The accompanying figure shows a sequence of different circuits. 

T2. 

(a) Solve for the current /j, for the circuit in part (a) of the figure. 

(b) Solve for the current / j, for the circuit in part (b) of the figure. 

(c) Solve for the current / j, for the circuit in part (c) of the figure. 




































(d) Continue this process until you discover a pattern in the values of [y 


(e) Investigate the sequence of values for in each of the circuits in parts (a), (b), (c), and so on, and numerically show 
that the limit of this sequence approaches the value 
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11.3 

GEOMETRIC LINEAR 
PROGRAMMING 


In this section we describe a geometric technique for maximizing or minimizing a 
linear expression in two variables subject to a set of linear constraints. 


Linear Systems 
Linear Inequalities 


Linear Programming 

The study of linear programming theory has expanded greatly since the pioneering work of George Dantzig in the late 1940s. 
Today, linear programming is applied to a wide variety of problems in industry and science. In this section we present a geometric 
approach to the solution of simple linear programming problems. Let us begin with some examples. 


Maximizing Sales Revenue 


A candy manufacturer has 130 pounds of chocolate-covered cherries and 170 pounds of chocolate-covered mints in stock. He 
decides to sell them in the form of two different mixtures. One mixture will contain half cherries and half mints by weight and will 
sell for $2.00 per pound. The other mixture will contain one-third cherries and two-thirds mints by weight and will sell for $1.25 
per pound. How many pounds of each mixture should the candy manufacturer prepare in order to maximize his sales revenue? 


Solution 

Let us first formulate this problem mathematically. Let the mixture of half cherries and half mints be called mix A, and let x \ be the 
number of pounds of this mixture to be prepared. Let the mixture of one-third cherries and two-thirds mints be called mix B, and 
let X 2 be the number of pounds of this mixture to be prepared. Since mix A sells for $2.00 per pound and mix B sells for $1.25 per 
pound, the total sales z (in dollars) will be 

z = 2.00xi + 1.25x2 

Since each pound of mix A contains ^ pound of cherries and each pound of mix B contains ^ pound of cherries, the total number 
of pounds of cherries used in both mixtures is 

^1 + ^2 

Similarly, since each pound of mix A contains ^ pound of mints and each pound of mix B contains j pound of mints, the total 
number of pounds of mints used in both mixtures is 

Because the manufacturer can use at most 130 pounds of cherries and 170 pounds of mints, we must have 

+ 1*2 <130 

^l + §*2 <170 

Furthermore, since xj and X 2 cannot be negative numbers, we must have 

xi >0 and X 2 > 0 

The problem can therefore be formulated mathematically as follows: Find values of xi and X 2 that maximize 

z = 2.00xi + 1.25x2 







subject to 


+ y *2 <130 
^1 + 1*2 <170 

xi > 0 
X2>0 

Later in this section we shall show how to solve this type of mathematical problem geometrically. 


Maximizing Annual Yield 


A woman has up to $10,000 to invest. Her broker suggests investing in two bonds, A and B. Bond A is a rather risky bond with an 
annual yield of 10%, and bond B is a rather safe bond with an annual yield of 7%. After some consideration, she decides to invest 
at most $6000 in bond A, to invest at least $2000 in bond B, and to invest at least as much in bond A as in bond B. How should she 
invest her $10,000 in order to maximize her annual yield? 


Solution 

To formulate this problem mathematically, let xi be the number of dollars to be invested in bond A, and let X 2 be the number of 
dollars to be invested in bond B. Since each dollar invested in bond A earns $.10 per year and each dollar invested in bond B earns 
$.07 per year, the total dollar amount z earned each year by both bonds is 

z= A0x\ + .07x2 

The constraints imposed can be formulated mathematically as follows: 

Invest no more than $10,000: *1 + X 2 < 10,000 

Invest at most $6000 in bond A: < 6000 

Invest at least $2000 in bond B : x 2 > 2000 

Invest at least as much in bond A as in bond B: x i > X 2 

We also have the implicit assumption that xj and X 2 are nonnegative: 

xi > 0 and X 2 > 0 

Thus the complete mathematical formulation of the problem is as follows: Find values of xj and X 2 that maximize 

z = .lOxi + 0.7x2 

subject to 

xi +X2 < 10,000 
xi< 6000 
x 2 > 2000 
xi —X2 > 0 
xi > 0 
X2 > 0 

♦ 


Minimizing Cost 





A student desires to design a breakfast of corn flakes and milk that is as economical as possible. On the basis of what he eats 
during his other meals, he decides that his breakfast should supply him with at least 9 grams of protein, at least the (RDA) of 

vitamin D, and at least the RDA of calcium. He finds the following nutrition information on the milk and corn flakes containers: 



Milk 

Corn Flakes 


(Vi cup) 

(1 ounce) 

Cost 

7.5 cents 

5.0 cents 

Protein 

4 grams 

2 grams 

Vitamin D 

1 of RDA 

— of RDA 

10 

Calcium 

— of RDA 

6 

None 


In order not to have his mixture too soggy or too dry, the student decides to limit himself to mixtures that contain 1 to 3 ounces of 
com flakes per cup of milk, inclusive. What quantities of milk and corn flakes should he use to minimize the cost of his breakfast? 


Solution 


For the mathematical formulation of this problem, let xj be the quantity of milk used (measured in -i-cup units), and let X 2 be the 
quantity of corn flakes used (measured in 1-ounce units). Then if z is the cost of the breakfast in cents, we may write the following. 

z = 7.5xi + 5.0x2 


Cost of breakfast: 

At least 9 grams protein: 


4xi + 2x2 ^ 9 


At least 4 RDA vitamin D: 
3 


8 ; ' 1 + 10^-3 


At least 4 RDA calcium: 
4 




At least 1 ounce corn flakes per cup (two 4--cups) of milk: 


(orxi -2 x 2 <0) 


At most 3 ounces com flakes per cup (two -i-cups) of milk: T2. < 2. ( or 3 ^. _ > 0) 

2 xi 2 v 1 J — " 

As before, we also have the implicit assumption that x j > o and X2 > 0- Thus the complete mathematical formulation of the 
problem is as follows: Find values of xj and X 2 that minimize 

z = 7.5xi + 5.0x2 
4xi 4 2x2 > 9 

h*4 

7 l i — 2*2 S 0 

3x\ — 2x2 ^ 0 
x\ > 0 
> 0 


subject to 







Geometric Solution of Linear Programming Problems 

Each of the preceding three examples is a special case of the following problem. 
Find values of jq and xq that either maximize or minimize 


subject to 


and 


Z = £ 1*1 


flliq 

+ 

^12*2 

a 2l x l 

+ 

a 22 x 2 

fljjjiq 

+ 

a rn2 x 2 


*1 > o. 


I £2*2 

<)(>)<=) *1 
<)(>)(=) h 

<)(>)(=) b m 

X2 > 0 


(1) 


( 2 ) 


( 3 ) 


In each of the m conditions of 2, any one of the symbols <, >, and = may be used. 

The problem above is called the general linear programming problem in two variables. The linear function z in 1 is called the 
objective function. Equations 2 and 3 are called the constraints ; in particular, the equations in 3 are called the nonnegativity 
constraints on the variables jq and jq- 

We shall now show how to solve a linear programming problem in two variables graphically. A pair of values (x xf) that satisfy 
all of the constraints is called a feasible solution. The set of all feasible solutions determines a subset of the qx 2 -plane called the 
feasible region. Our desire is to find a feasible solution that maximizes the objective function. Such a solution is called an optimal 
solution. 

To examine the feasible region of a linear programming problem, let us note that each constraint of the form 

-1*1 H a i2 x 2 = bi 

defines a line in the qjq-plane, whereas each constraint of the form 

anx i +aj 2 X 2 <bi or a,- 1*1 + a^ x 2 >bj 

defines a half-plane that includes its boundary line 

£ti\x\ H a i2 X2 = bi 

Thus the feasible region is always an intersection of finitely many lines and half-planes. For example, the four constraints 

jxi + ^2 <130 

^q + |*2 <170 
jq > 0 
X2 > 0 

of Example 1 define the half-planes illustrated in parts (a), (b), (c), and (cl) of Figure 11.3.1. The feasible region of this problem is 
thus the intersection of these four half-planes, which is illustrated in Figure 11.3.1 e. 
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Figure 11.3.1 

It can be shown that the feasible region of a linear programming problem has a boundary consisting of a finite number of straight 
line segments. If the feasible region can be enclosed in a sufficiently large circle, it is called bounded (Figure 11.3.1e); otherwise, 
it is called unbounded (see Figure 11.3.5). If the feasible region is empty (contains no points), then the constraints are inconsistent 
and the linear programming problem has no solution (see Figure 11.3.6). 

Those boundary points of a feasible region that are intersections of two of the straight line boundary segments are called extreme 
points. (They are also called corner points and vertex points.) For example, in Figure 11,3.1c, we see that the feasible region of 
Example 1 has four extreme points: 


(0, 0), (0, 255), (180, 120), (260, 0) 


(4) 


The importance of the extreme points of a feasible region is shown by the following theorem. 

THEOREM 11.3.1 


Maximum and Minimum Values 

If the feasible region of a linear programming problem is nonempty and bounded, then the objective function attains both a 
maximum and a minimum value, and these occur at extreme points of the feasible region. If the feasible region is unbounded, 
then the objective function may or may not attain a maximum or minimum value; however, if it attains a maximum or minimum 
value, it does so at an extreme point. 


Figure 11.3.2 suggests the idea behind the proof of this theorem. Since the objective function 

Z = C\X\ +c 2X2 

of a linear programming problem is a linear function of x\ and ?:- A , its level curves (the curves along which z has constant values) 
are straight lines. As we move in a direction perpendicular to these level curves, the objective function either increases or decreases 
monotonically. Within a bounded feasible region, the maximum and minimum values of z must therefore occur at extreme points, 
as Figure 11.3.2 indicates. 















Figure 11.3.2 


In the next few examples we use Theorem 11.3.1 to solve several linear programming problems and illustrate the variations in the 
nature of the solutions that may occur. 


EXAMPLE 4 Example 1 Revisited 


Figure ll.3.le shows that the feasible region of Example 1 is bounded. Consequently, from Theorem 11.3.1 the objective function 

z = 2.00xi + 1-25x2 

attains both its minimum and maximum values at extreme points. The four extreme points and the corresponding values of z are 
given in the following table. 


Extreme Point 

Value of 

(*1. *2) 

z = 2.00xi T- 1.25x2 

(0,0) 

0 

(0, 255) 

318.75 

(180, 120) 

510.00 

(260, 0) 

520.00 


We see that the largest value of z is 520.00 and the corresponding optimal solution is (260, 0). Thus the candy manufacturer attains 
maximum sales of $520 when he produces 260 pounds of mixture A and none of mixture B. 


EXAMPLE 5 Using Theorem 11.3.1 


Find values of xj and X 2 that maximize 


subject to 


z = x i 4 3x2 












Solution 


2x\ f 3 x 2 < 24 
x\-X2 <7 
*2 < 6 
jq > 0 
X2 > 0 


In Figure 11.3.3 we have drawn the feasible region of this problem. Since it is bounded, the maximum value of z is attained at one 
of the five extreme points. The values of the objective function at the five extreme points are given in the following table. 



Extreme Point 

(* 1 . * 2 ) 

Value of 

z = x 1 + 3 x 2 

( 0 , 6 ) 

18 

(3, 6 ) 

21 

(9, 2 ) 

15 

(7, 0 ) 

7 

( 0 , 0 ) 

0 


From this table, the maximum value of z is 21, which is attained at =3 and ^ = 6 - 


♦ 


Using Theorem 11.3.1 


Find values of jq and X2 that maximize 


subject to 


z = Ax\ 4- 6 x 2 









Solution 


2x\ f3x2<24 
x\-X 2 <7 
*2 < 6 
jq > 0 
X2 > 0 


The constraints in this problem are identical to the constraints in Example 5, so the feasible region of this problem is also given by 
Figure 11.3.3. The values of the objective function at the extreme points are given in the following table. 


Extreme Point 

(* 1 . * 2 ) 

Value of 

z = Ax\ + 6 x 2 

( 0 , 6 ) 

36 

(3, 6 ) 

48 

(9, 2 ) 

48 

(7, 0 ) 

28 

( 0 , 0 ) 

0 


We see that the objective function attains a maximum value of 48 at two adjacent extreme points, (3, 6 ) and (9, 2). This shows that 
an optimal solution to a linear programming problem need not be unique. As we ask the reader to show in Exercise 10, if the 
objective function has the same value at two adjacent extreme points, it has the same value at all points on the straight line 
boundary segment connecting the two extreme points. Thus, in this example the maximum value of z is attained at all points on the 
straight line segment connecting the extreme points (3, 6 ) and (9, 2). 


The Feasible Region Is a Line Segment 


Find values of xj and X 2 that minimize 
subject to 


z = 2x\ —X 2 

+ 3x2 =12 
2xi — 3x2 > 0 
xi > 0 
X2> 0 


Solution 

In Figure 11.3.4 we have drawn the feasible region of this problem. Because one of the constraints is an equality constraint, the 
feasible region is a straight line segment with two extreme points. The values of z at the two extreme points are given in the 
following table. 










Extreme Point 

Value of 

Oi. * 2 ) 

z = 2xi — X 2 

(3, 2) 

4 

(6, 0) 

12 


The minimum value of z is thus 4 and is attained at x j = 3 and X2 = 2- 


♦ 


Using Theorem 11.3.1 

Find values of jq and X 2 that maximize 

subject to 


z = 2xi 4 5^2 

2xi + X 2 > 8 
— 4xi 4- X 2 < 2 
2xi — 3x2 < 0 
xi > 0 
X2 > 0 


Solution 

The feasible region of this linear programming problem is illustrated in Figure 11.3.5. Since it is unbounded, we are not assured by 
Theorem 11.3.1 that the objective function attains a maximum value. In fact, it is easily seen that since the feasible region contains 
points for which both jq and X 2 are arbitrarily large and positive, the objective function 

z = 2xi + 5x2 

can be made arbitrarily large and positive. This problem has no optimal solution. Instead, we say the problem has an unbounded 
solution. 












♦ 


Using Theorem 11.3.1 

Find values of xj and jq that maximize 

subject to 


z = — 5xi f *2 


2x i + 

*2 > 8 

-4xi + 

*2 "5 2 

2xi — 

3x 2 < 0 

*1 > 

0 

Al 

r-i 

0 


Solution 


The above constraints are the same as those in Example 8 , so the feasible region of this problem is also given by Figure 11.3.5. In 
Exercise 11 we ask the reader to show that the objective function of this problem attains a maximum within the feasible region. By 
Theorem 11.3.1, this maximum must be attained at an extreme point. The values of z at the two extreme points of the feasible 
region are given in the following table. 


Extreme Point 

Value of 

(* 1 . * 2 ) 

z= — 5xi + X 2 

( 1 , 6 ) 

1 

(3,2) 

-13 


The maximum value of z is thus 1 and is attained at the extreme point q = 1 , q = 6 . 


♦ 


Inconsistent Constraints 


Find values of q and q that minimize 


z = 3x\ — 8 x 2 










subject to 


2xi — 

*2 


4 

3xi F 

11x2 

< 

33 

3xi + 

4x2 

> 

24 


*1 

> 

0 


X2 

> 

0 


Solution 

As can be seen from Figure 11.3.6, the intersection of the five half-planes defined by the five constraints is empty. This linear 
programming problem has no feasible solutions since the constraints are inconsistent. 



Figure 11.3.6 

There are no points common to all five shaded half-planes. 


Exercise Set 11.3 



Click here for Just Ask! 


Find values of xi and X 2 that maximize 
z=3x\ 2x2 

subject to 

2xi + 3x2 < 6 
2xi — X 2 > 0 
xi <2 
*2 < 1 
xi > 0 
X2 > 0 


Find values of xi and X 2 that minimize 
z = 3xi — 5^2 


subject to 







2x\ — — 2 

4x\ —X 2 > 0 

*2 < 3 

x\ > 0 

*2 > 0 

Find values of jc j and X 2 that minimize 
z = — 3x\ 4 2j:2 

subject to 

3*i - *2 > - 5 
-*l+*2> 1 

2*1 4 %2 > 12 
*1 > 0 

*2 > 0 

Solve the linear programming problem posed in Example 2. 

4. 


Solve the linear programming problem posed in Example 3. 

5. 

In Example 5 the constraint ;q _ X2 < 7 is said to be nonbinding because it can be removed from the problem without affecting 
”• the solution. Likewise, the constraint *2 < 6 is said to be binding because removing it will change the solution. 

(a) Which of the remaining constraints are nonbinding and which are binding? 

(b) For what values of the righthand side of the nonbinding constraint x j — X 2 < 7 will this constraint become binding? For 
what values will the resulting feasible set be empty? 

(c) For what values of the righthand side of the binding constraints X2 < 6 will this constraint become nonbinding? For 
what values will the resulting feasible set be empty? 


A trucking firm ships the containers of two companies, A and B. Each container from company A weighs 40 pounds and is 2 
7- cubic feet in volume. Each container from company B weighs 50 pounds and is 3 cubic feet in volume. The trucking firm 
charges company A $2.20 for each container shipped and charges company B $3.00 for each container shipped. If one of the 
firm's trucks cannot carry more than 37,000 pounds and cannot hold more than 2000 cubic feet, how many containers from 
companies A and B should a truck carry to maximize the shipping charges? 

Repeat Exercise 7 if the trucking firm raises its price for shipping a container from company A to $2.50. 

8 . 

A manufacturer produces sacks of chicken feed from two ingredients, A and B. Each sack is to contain at least 10 ounces of 
9. nutrient Ji/j, at least 8 ounces of nutrient }j 2 , and at least 12 ounces of nutrient Ny Each pound of ingredient A contains 2 
ounces of nutrient 2 ounces of nutrient ]\[ 2 , and 6 ounces of nutrient $Jy Each pound of ingredient B contains 5 ounces of 
nutrient jyj, 3 ounces of nutrient ]\[ 2 , and 4 ounces of nutrient If ingredient A costs 8 cents per pound and ingredient B 
costs 9 cents per pound, how much of each ingredient should the manufacturer use in each sack of feed to minimize his costs? 



If the objective function of a linear programming problem has the same value at two adjacent extreme points, show that it has 
10 * the same value at all points on the straight line segment connecting the two extreme points. 

Hint If ( x [, x' 2 ) and (x" 1 , x” 2 ) are any two points in the plane, a point (x\, x 2 ) '' es on the straight line segment connecting 
them if 


and 


XI =tx ! 1 + (1 — t)x H 1 

X2 = tx ! 2 + (1 -t)x >! 2 


where t is a number in the interval [0, 1]. 


Show that the objective function in Example 10 attains a maximum value in the feasible set. 

11 . 

Hint Examine the level curves of the objective function. 
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Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Tl. 


Consider the feasible region consisting of 0 < x, 0 < y a l°ng with the set of inequalities 


X cos 


(2k f l)g 
An 


+ y sin 


(2k f l)x 
An 


< cos 


(i) 


for fc = 0 , 1,2, ...,«_ 1 . Maximize the objective function 

z=3x A- Ay 


assuming that (a) n = 1 , (b) n = 2 , (c) n = 3 , (d) n = 4 , (e) n = 5 , (f) n = g, (g) n = 7 , (h) n = g, (i) n = 9 , (j) n = 10 , and (k) 
n = 11- (1) Next, maximize this objective function using the nonlinear feasible region, 0 < x , 0 <y, and 


(m) Let the results of parts (a) through (k) begin a sequence of values for Do these values approach the value determined 
in part (1)? Explain. 


Repeat Exercise Tl using the objective function z = x + y. 

T2. 
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11.4 

THE EARLIEST 
APPLICATIONS OF 
LINEAR ALGEBRA 


Linear systems can be found in the earliest writings of many ancient 
civilizations. We give some examples of the types of problems that they used to 
solve. 


Linear Systems 

The practical problems of early civilizations included the measurement of land, the distribution of goods, the tracking of resources 
such as wheat and cattle, and taxation and inheritance calculations. In many cases, these problems led to linear systems of 
equations since linearity is one of the simplest relationship that can exist among variables. In this section we present examples 
from five diverse ancient cultures illustrating how they used and solved systems of linear equations. We restrict ourselves to 
examples before a.d. 500. These examples consequently predate the development, by Islamic/Arab mathematicians, of the field of 
algebra, a field that ultimately led, in the nineteenth century, to the branch of mathematics now called linear algebra. 


Egypt (about 1650 b.c.) 



Problem 40 of the Ahmes Papyrus 


The Ahmes (or Rhind) Papyrus is the source of most of our information about ancient Egyptian mathematics. This 5-meter-long 
papyrus contains 84 short mathematical problems, together with their solutions, and dates from about 1650 B.C. Problem 40 in this 
papyrus is the following: 


Divide 100 hekats of barley among five men in arithmetic progression so that the sum of the two smallest is one-seventh the 
sum of the three largest. 


Let a be the least amount that any man obtains, and let d be the common difference of the terms in the arithmetic progression. Then 

the other four men receive a 4 d, a 4- 2d, a 4- 3d, and a A-Ad hekats. The two conditions of the problem require that 

a 4- {a A- d) 4- (a 4 2d) 4- (a 4 3d) 4- (a I Ad) = 100 

[ (a 4 2d) 4- (a 4 3d) 4- (a 4 Ad) =aA-{aA-d) 

These equations reduce to the following system of two equations in two unknowns: 

5a 4 10tf = 100 

Iks —2^ = 0 W 

The solution technique described in the papyrus is known as the method of false position or false assumption. It begins by 
assuming some convenient value of a (in our case a = 1), substituting that value into the second equation, and obtaining d = \\! 2- 
Substituting a = 1 and d = 11/2 into the left-hand side of the first equation gives 60, whereas the right-hand side is 100. 









Adjusting the initial guess for a by multiplying it by 100/60 leads to the correct value a = 5 I 3- Substituting a = 5 / 3 into the 
second equation then gives d = 55 / 6, so the quantities of barley received by the five men are 10/6, 65/6, 120/6, 175/6, and 230/6 
hekats. This technique of guessing a value of an unknown and later adjusting it has been used by many cultures throughout the 
ages. 

♦ 


Babylonia (1900-1600 b.c.) 



Babylonian Clay Tablet Ca MLA 1950 


The Old Babylonian Empire flourished in Mesopotamia between 1900 and 1600 B.c. Many clay tablets containing mathematical 
tables and problems survive from that period, one of which (designated Ca MLA 1950) contains the next problem. The statement 
of the problem is a bit muddled because of the condition of the tablet, but the diagram and the solution on the tablet indicate that 
the problem is as follows: 



A trapezoid with an area of 320 square units is cut off from a right triangle by a line parallel to one of its sides. The other side 
has length 50 units, and the height of the trapezoid is 20 units. What are the upper and the lower widths of the trapezoid? 


Let x be the lower width of the trapezoid and y its upper width. The area of the trapezoid is its height times its average width, so 
20 f — ^ — | = 320. Using similar triangles, we also have = -j^-. The solution on the tablet uses these relations to generate the 


linear system 


















(2) 


!(*+;,) = 16 
±(x-y)=4 

Adding and subtracting these two equations then gives the solution x = 20 and y = 12- 


China (a.d. 263) 




Chiu Chang Suan Shu in Chinese characters 


The most important treatise in the history of Chinese mathematics is the Chiu Chang Suan Shu, or “The Nine Chapters of the 
Mathematical Art.” This treatise, which is a collection of 246 problems and their solutions, was assembled in its final form by Liu 
Hui in A.D. 263. Its contents, however, go back to at least the beginning of the Han dynasty in the second century B.C. The eighth of 
its nine chapters, entitled “The Way of Calculating by Arrays,” contains 18 word problems that lead to linear systems in three to 
six unknowns. The general solution procedure described is almost identical to the Gaussian elimination technique developed in 
Europe in the nineteenth century by Carl Friedrich Gauss (see page 13 for his biography). The first problem in the eighth chapter is 
the following: 


There are three classes of corn, of which three bundles of the first class, two of the second, and one of the third make 39 
measures. Two of the first, three of the second, and one of the third make 34 measures. And one of the first, two of the second, 
and three of the third make 26 measures. How many measures of grain are contained in one bundle of each class? 


Let x, y, and z be the measures of the first, second, and third classes of corn. Then the conditions of the problem lead to the 
following linear system of three equations in three unknowns: 

3x 4- 2y + z = 39 
2x f 3y I z = 34 
x + 2y A 3z — 26 


The solution described in the treatise represented the coefficients of each equation by an appropriate number of rods placed within 
squares on a counting table. Positive coefficients were represented by black rods, negative coefficients were represented by red 
rods, and the squares corresponding to zero coefficients were left empty. The counting table was laid out as follows so that the 
coefficients of each equation appear in columns with the first equation in the rightmost column: 


1 

2 

3 

2 

3 

2 

3 

1 

1 

26 

34 

39 


Next the numbers of rods within the squares were adjusted to accomplish the following two steps: 1 two times the numbers of the 
third column were subtracted from three times the numbers in the second column and 2 the numbers in the third column were 
subtracted from three times the numbers in the first column. The result was the following array: 











In this array, four times the numbers in the second column were subtracted from five times the numbers in the first column, 
yielding 


3 


5 2 


36 1 1 


99 24 39 

This last array is equivalent to the linear system 

3x + 2y 4 z = 39 
5y -\- z = 2A 
36z = 99 

This triangular system was solved by a method equivalent to back-substitution to obtain x = 37 / 4 , y = 17 / 4 , and z = 11 / 4- 


EXAMPLE 4 Greece (third century b.c.) 



Archimedes c. 287-212 B.c. 


Perhaps the most famous system of linear equations from antiquity is the one associated with the first part of Archimedes’ 
celebrated Cattle Problem. This problem supposedly was posed by Archimedes as a challenge to his colleague Eratosthenes. No 
solution has come down to us from ancient times, so that it is not known how, or even whether, either of these two geometers 
solved it. 















If thou art diligent and wise, O stranger, compute the number of cattle of the Sun, who once upon a time grazed on the fields of 
the Thrinacian isle of Sicily, divided into four herds of different colors, one milk white, another glossy black, a third yellow, and 
the last dappled. In each herd were bulls, mighty in number according to these proportions: Understand, stranger, that the 
white bulls were equal to a half and a third of the black together with the whole of the yellow, while the black were equal to the 
fourth part of the dappled and a fifth, together with, once more, the whole of the yellow. Observe further that the remaining 
bulls, the dappled, were equal to a sixth part of the white and a seventh, together with all of the yellow. These were the 
proportions of the cows: The white were precisely equal to the third part and a fourth of the whole herd of the black; while the 
black were equal to the fourth part once more of the dappled and with it a fifth part, when all, including the bulls, went to 
pasture together. Now the dappled in four parts were equal in number to a fifth part and a sixth of the yellow herd. Finally the 
yellow were in number equal to a sixth part and a seventh of the white herd. If thou canst accurately tell, O stranger, the 
number of cattle of the Sun, giving separately the number of well-fed bulls and again the number of females according to each 
color, thou wouldst not be called unskilled or ignorant of numbers, but not yet shalt thou be numbered among the wise. 


The conventional designation of the eight variables in this problem is 

W = number of white bulls 
B = number of black bulls 

Y = number of yellow bulls 

D = number of dappled bulls 

w = number of white cows 
b = number of black cows 

V = number of yellow cows 

d = number of dappled cows 

The problem can now be stated as the following seven homogeneous equations in eight unknowns: 

(The white bulls were equal to a half and a third of the black [bulls] together with the whole of 
the yellow [bulls].) 

(The black [bulls] were equal to the fourth part of the dappled [bulls] and a fifth, together with, 
once more, the whole of the yellow [bulls].) 

(The remaining bulls, the dappled, were equal to a sixth part of the white [bulls] and a seventh, 
together with all of the yellow [bulls].) 

(The white [cows] were precisely equal to the third part and a fourth of the whole herd of the 
black.) 

(The black [cows] were equal to the fourth part once more of the dappled and with it a fifth part, 
when all, including the bulls, went to pasture together.) 

(The dappled [cows] in four parts [that is, in totality] were equal in number to a fifth part and a 
sixth of the yellow herd.) 

(The yellow [cows] were in number equal to a sixth part and a seventh of the white herd.) 

As we ask the reader to show in the exercises, this system has infinitely many solutions of the form 


1 . w=(± + ±)B+Y 

2- 5=(± + l)D+r 
3. D=(l + l)r + y 
4 - ™ = + 

5 ‘ £ = (^ + ^)(D + ^) 

6. ^ = (i + i)(7f i y) 

7 ‘ y = + 


(4) 


W= 10,366,482^: 

B = 7,460,514* 

7=4,149,387* 

D = 7,358,060* 
w = 7,206,360* 

* = 4,893,246* 
j = 5,439,213* 
d = 3,515,820* 

where * is any real number. The values * = 1 , 2, ... give infinitely many positive integer solutions to the problem, with * = 1 

giving the smallest solution. 


♦ 


EXAM India (fourth century a.d.) 



Fragment lll-5-3v of the Bakhshali Manuscript 


The Bakhshali Manuscript is an ancient work of Indian/Hindu mathematics dating from around the fourth century A.D., although 
some of its materials undoubtedly come from many centuries before. It consists of about 70 leaves or sheets of birch bark 
containing mathematical problems and their solutions. Many of its problems are so-called equalization problems that lead to 
systems of linear equations. One such problem on the fragment shown is the following: 


One merchant has seven asava horses, a second has nine hay a horses, and a third has ten camels. They are equally well off in 
the value of their animals if each gives two animals, one to each of the others. Find the price of each animal and the total value 
of the animals possessed by each merchant. 


Let x be the price of an asava horse, let y be the price of a haya horse, let z be the price of a camel, and the let K be the total value 
of the animals possessed by each merchant. Then the conditions of the problem lead to the following system of equations: 

5x4 -y+z = K 

x \-ly+z = K ( 5 ) 

x +y-\- Ez = K 

The method of solution described in the manuscript begins by subtracting the quantity (x \ y \ z) from both sides of the three 
equations to obtain Ax = 6y = lz = K — {x + y -V z)- This shows that if the prices x, y, and z are to be integers, then the quantity 
K — (x + y + z) must be an integer that is divisible by 4, 6, and 7. The manuscript takes the product of these three numbers, or 
168, for the value £ _ (* 4. y 4. z ), which yields x = 42, y = 28> and z = 24 for the prices and K = 262 for the total value. (See 
Exercise 6 for more solutions to this problem.) 

♦ 




Exercise Set 11.4 



Click here for Just Ask! 


The following lines from Book 12 of Homer's Odyssey relate a precursor of Archimedes’ Cattle Problem: 

1 . 

Thou shalt ascend the isle triangular. 

Where many oxen of the Sun are fed, 

And fatted flocks. Of oxen fifty head 


In every herd feed, and their herds are seven; 


And of his fat flocks is their number even. 

The last line means that there are as many sheep in all the flocks as there are oxen in all the herds. What is the total number of 
oxen and sheep that belong to the god of the Sun? (This was a difficult problem in Homer's day.) 

Solve the following problems from the Bakhshali Manuscript. 

2 . 


(a) B possesses two times as much as A; C has three times as much as A and B together; D has four times as much as A, B, 
and C together. Their total possessions are 300. What is the possession of A? 


(b) B gives 2 times as much as A; C gives 3 times as much as B; D gives 4 times as much as C. Their total gift is 132. What 
is the gift of A? 


A problem on a Babylonian tablet requires finding the length and width of a rectangle given that the length and the width add 
3- up to 10, while the length and one-fourth of the width add up to 7. The solution provided on the tablet consists of the following 
four statements: 

Multiply 7 by 4 to obtain 28. 


Take away 10 from 28 to obtain 18. 


Take one-third of 18 to obtain 6, the length. 


Take away 6 from 10 to obtain 4, the width. 

Explain how these steps lead to the answer. 

The following two problems are from “The Nine Chapters of the Mathematical Art.” Solve them using the array technique 
4. described in Example 3. 



(a) Five oxen and two sheep are worth 10 units and two oxen and five sheep are worth 8 units. What is the value of each ox 
and sheep? 

(b) There are three kinds of corn. The grains contained in two, three, and four bundles, respectively, of these three classes 
of corn, are not sufficient to make a whole measure. However, if we added to them one bundle of the second, third, and 
first classes, respectively, then the grains would become on full measure in each case. How many measures of grain 
does each bundle of the different classes contain? 


This problem in part (a) is known as the “Flower of Thymaridas,” named after a Pythagorean of the fourth century B.c. 

(a) Given the n numbers ct\,ct 2 , ...» a„, solve for jq, ..., x,, in the following linear system: 

*1 F *2 H-b x H =a\ 

*1 -b *2 = a 2 

xi -b X 3 =fl3 
xi 4- x H =a n 

(b) Identify a problem in this exercise set that fits the pattern in part (a), and solve it using your general solution. 


For Example 5 from the Bakhshali Manuscript: 

(a) Express Equations 5 as a homogeneous linear system of three equations in four unknowns (. x , y, z, and K) and show that 
the solution set has one arbitrary parameter. 


(b) Find the smallest solution for which all four variables are positive integers. 


(c) Show that the solution given in Example 5 is included among your solutions. 


Solve the problems posed in the following three epigrams, which appear in a collection entitled “The Greek Anthology,” which 
was compiled in part by a scholar named Metrodorus around A.D. 500. Some of its 46 mathematical problems are believed to 
date as far back as 600 B.c. (Before solving parts (a) and (c), you will have to formulate the question.) 


(a) I desire my two sons to receive the thousand staters of which I am possessed, but let the fifth part of the legitimate one's 
share exceed by ten the fourth part of what falls to the illegitimate one. 

(b) Make me a crown weighing sixty minae, mixing gold and brass, and with them tin and much-wrought iron. Let the gold 
and brass together form two-thirds, the gold and tin together three-fourths, and the gold and iron three-fifths. Tell me 
how much gold you must put in, how much brass, how much tin, and how much iron, so as to make the whole crown 
weigh sixty minae. 

(c) First person: I have what the second has and the third of what the third has. Second person: I have what the third has 
and the third of what the first has. Third person: And I have ten minae and the third of what the second has. 


Section 11.4 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Tl. 

(a) Solve Archimedes’ Cattle Problem using a symbolic algebra program. 


(b) The Cattle Problem has a second part in which two additional conditions are imposed. The first of these states that 
“When thewhite bulls mingled their number with the black, they stood firm, equal in depth and breadth.” This requires 
that W + B be a square number, i.e., 1, 4, 9, 16, 25, etc. Show that this requires that the values of k in Eq. 4 be 
restricted as follows: 


k = 4,456J49r 2 , r= 1,2,3, ... 

and find the smallest total number of cattle that satisfies this second condition. 


Remark The second condition imposed in the second part of the Cattle Problem states that “When the yellow and the 
dappled bulls were gathered into one herd, they stood in such a manner that their number, beginning from one, grew slowly 
greater 'til it completed a triangular figure.” This requires that the quantity Y4 - L be a triangular number—that is, a number of 
the form 1,1+2, 1+2 + 3,1+2 + 3 +4, .... This final part of the problem was not completely solved until 1965 when all 
206,545 digits of the smallest number of cattle that satisfies this condition were found using a computer. 


The following problem is from “The Nine Chapters of the Mathematical Art” and determines a homogeneous linear system 
five equations in six unknowns. Show that the system has infinitely many solutions, and find the one for which the depth o 
the well and the lengths of the five ropes are the smallest possible positive integers. 


Suppose that five families share a well. Suppose further that 


2 of A's ropes are short of the well's depth by one of B's ropes. 


3 of B's ropes are short of the well's depth by one of C's ropes. 


4 of C's ropes are short of the well's depth by one of D's ropes. 


5 of D's ropes are short of the well's depth by one of E's ropes. 


6 of E's ropes are short of the well's depth by one of A's ropes. 
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11.5 

CUBIC SPLINE 
INTERPOLATION 


In this section an artist's drafting aid is used as a physical model for the 
mathematical problem of finding a curve that passes through specified points in 
the plane. The parameters of the curve are determined by solving a linear 
system of equations. 


Linear Systems 
Matrix Algebra 
Differential Calculus 


Curve Fitting 

Fitting a curve through specified points in the plane is a common problem encountered in analyzing experimental data, in 
ascertaining the relations among variables, and in design work. In Figure 11.5.3 seven points in the xy-plane are displayed, and in 
Figure 11.5.2 a smooth curve has been drawn that passes through them. A curve that passes through a set of points in the plane is 
said to interpolate those points, and the curve is called an interpolating curve for those points. The interpolating curve in Figure 
11.5.1 was drawn with the aid of a drafting spline (Figure 11.5.2). This drafting aid consists of athin, flexible strip of wood or 
other material that is bent to pass through the points to be interpolated. Attached sliding weights hold the spline in position while 
the artist draws the interpolating curve. The drafting spline will serve as the physical model for a mathematical theory of 
interpolation that we will discuss in this section. 



Figure 11.5.1 



Figure 11.5.2 







-T 


A -' 1 


Figure 11.5.3 


Statement of the Problem 

Suppose that we are given n points in the xy-plane, 

Ol. 7l). (*2. y2),---, Oh, y n ) 

which we wish to interpolate with a “well-behaved” curve (Figure 11.5.4). For convenience, we take the points to be equally 
spaced in the x-direction, although our results can easily be extended to the case of unequally spaced points. If we let the common 
distance between the x-coordinates of the points be h, then we have 

*2 -*1 =*3 -*2 = - = x„ -*h -1 =h 

Let y = S(x), xi < ?; < x denote the interpolating curve that we seek. We assume that this curve describes the displacement of a 
drafting spline that interpolates the n points when the weights holding down the spline are situated precisely at the n points. It is 
known from linear beam theory that for small displacements, the fourth derivative of the displacement of a beam is zero along any 
interval of the x-axis that contains no external forces acting on the beam. If we treat our drafting spline as a thin beam and realize 
that the only external forces acting on it arise from the weights at the n specified points, then it follows that 

£ (lv) 0) = 0 m 


for values of x lying in the n — \ open intervals 

(xi, x 2 ), (x 2 , x 3 )..., Oh- 1, x H ) 


between the n points. 



We also need the result from linear beam theory that states that for a beam acted upon only by external forces, the displacement 
must have two continuous derivatives. In the case of the interpolating curve y = S(x') constructed by the drafting spline, this means 
that S(x)> ^O), and £ w (x) must be continuous for jq < x < x H - 

The condition that S !/ (x) be continuous is what causes a drafting spline to produce a pleasing curve, as it results in continuous 
curvature. The eye can perceive sudden changes in curvature—that is, discontinuities in S'" (x)—but sudden changes in higher 
derivatives are not discernible. Thus, the condition that S !! (x ) be continuous is the minimal prerequisite for the interpolating curve 
to be perceptible as a single smooth curve, rather than as a series of separate curves pieced together. 

To determine the mathematical form of the function S(x), we observe that because S^(x) 0 in the intervals between the n 

specified points, it follows by integrating this equation four times that S(x) must be a cubic polynomial in x in each such interval. 
In general, however, S(x) will be a different cubic polynomial in each interval, so Six) must have the form 






S(x) = 


Site), 


?i i < x < ^2 
*2 ^ x *3 


( 2 ) 


Syi — lOO. 1 S 1 ^ 


where S\ (*), S 2 (x)’ • • •> S n -\ (x) are cu bic polynomials. For convenience, we will write these in the form 

3 2 

S\ (x) = a\ (x — ;q) -h A i (jt — xi) 4 ci (x — *i) 4 < 3 ?i, xi<x<x 2 

*2 <* <*2 


Site) =aite-X 2 ) 2 4 b 2 {x ~ *l) 2 4 cite-X 2 ) 4 d 2 . 


(3) 


Sy\ —l(^ 0 — a n —1 te — l) I byi —i ix Xyi —i) | c H _i (j — j h _i) | d n — i. 


X _i < x < x n 


The aj/s, bjf s, c,-/s, and dps constitute a total of An — A coefficients that we must determine to specify Six ) completely. If we 
choose these coefficients so that Six) interpolates the n specified points in the plane and Six), S'(x), and S >f (x) are continuous, 
then the resulting interpolating curve is called a cubic spline. 

Derivation of the Formula of a Cubic Spline 


From Equations 2 and 3, we have 


S(x) =Si(x) =a lte — ^ri) 3 4 — ^i) 2 4ci(i- *i) 4<a?i, 

S(x) = S 2 (x) =G 2 te -* 2 ) 3 + hte-xi) 2 4 c 2 (x ~ *l) + d 2 , 

S(x) =S n -i(x) = ««-!(* =*«-l) 3 4 b n -\ (x — Xyi-i) 2 H + 


x\ < x < ^2 
^2 ^ ^ 

x H _i < x < x n 


(4) 


so 


S'(x) =S'\(x) = 3a\(x —x\) 2 4 2b\(x — x\) 4 c\, 
s'(x) =s'2(x) = 3a2te-X2) 2 4 2b 2 (x -x 2 ) 4 c 2 . 


X 1 < X < X 2 

X2—-X-- *3 


(5) 


S te) — S yi —i ix) — 3a n —\ (x — x M —l) 4 2i H _i (x — x H —l) 4 c H —1> 


and 


S"(x)=S" i(i) = 6 fli(jr-ii) + 2ii. 
S"(x)=S" 2 te) =6a 2 (x-x 2 ) + 2b 2 . 


X\ <X<X 2 
^2 ^ X ^ 7T3 


( 6 ) 


S"te)=S lf n-ite) = ^-lte-Xy l -l) 4 2 b n -i, *„_i <*<*„ 


We will now use these equations and the four properties of cubic splines stated below to express the unknown coefficients a,-, £., 
Cj, dj, j = 1 , 2 , 1 , in terms of the known coordinates y l5 y-j, ..., y„. 


interpolates the points (x^, _y,), ? = 1 , 2, 


n. Because £(*) interpolates the points . f y t ), j = 1 , 2, ..., n, we have 


£(*13=71. £(*23 =72.---. ^h)=7b 


(V) 


From the first n — 1 of these equations and 4, we obtain 

<*1 =71 

di=yi 
dyi — 1 = 7»— 1 

From the last equation in 7, the last equation in 4, and the fact that x „ -x n -\ = h, we obtain 

4 b n _\h 2 4 c„_iA 4 d n -\ = y„ 


( 8 ) 


(9) 




S(x) is continuous on [xi, x n ]- Because Six) is continuous for x i < x < x n , it follows that at each point x, in the set x 2 


*3 x H _i we must have 



Si-\{^i)=S i {x i ), i = 2, 3 ,1 (1() j 


Otherwise, the graphs of S',_ 1 (^) and Sj(x) would not join together to form a continuous curve at x,. 
When we apply the interpolating property Sj(xj) =y u it follows from 10 that £,_!(>:,) =y u i = 2, 3, 
n - 1, or from 4 that 



3 2 

a\h -\-b\h + c\k+d\=y2 

3 2 

a 2 h -V b 2 h A-c 2 hAd 2 =y 2 

a n - 2 h 2 A- b n - 2 h 2 A- c n _ 2 h A- d n _ 2 = y n - 1 

S ! {x) is continuous on [x\, x H ]. 

Because S ! (x) is continuous for < x < x n , it follows that 

or, from 5, 

S , i -i(x i )=S , i (x i ), i = 2, 3,..., M-l 

3a\h 2 A 2b\hA-c\=c 2 

3a 2 h 2 A- 2b 2 h A- c 2 = c 2 

2 

3a n - 2 h 4- 2b Yl - 2 k 4- c H _ 2 = c H _ i 

4, S !! (x) is continuous on [x 1; x 2 ]- 

Because S ff (x ) is continuous for x i < x < x n , it follows that 

or, from 6, 

S" i - X {x i )=S' , i {x i ), i = 2, 3. M-l 

6a 1/2 + 2Z?i = 2b 2 

6a?k +■ 2b? = 2b? 

5 (13) 

&&yi— 2^ 1 2by}— 2 — 2by} — \ 


Equations 8,9, 11, 12, and 13 constitute a system of 4« _ 6 linear equations in the 4« — 4 unknown coefficients a,, h v cp dp i = 1, 
2, — l. Consequently, we need two more equations to determine these coefficients uniquely. Before obtaining these additional 

equations, however, we can simplify our existing system by expressing the unknowns ap bp cp and dj in terms of new unknown 
quantities 

Mi = M 2 = S"(x 2 ),.... M» = $"(x») 


and the known quantities 

yu y?.,—, y» 

For example, from 6 it follows that 

Afl = 2*i 

M 2 = 2b 2 

M n —\ = 2b n —\ 


b 2 = \u 2 ,-., b n ^=^M n ^ 


SO 


Moreover, we already know from 8 that 

<*\=y\, d 2 =y 2 ,..., d n -i=yn-\ 

We leave it as an exercise for the reader to derive the expressions for the a,-/s and c,7s in terms of the Mj/s and s. The final 
result is as follows: 

THEOREM 11.5.1 


Cubic Spline Interpolation 




Given n points 7l ), (x 2 , y 2 ) 

Sr- 

+ 

1 

II 

Sr- 

if 

K> 

£ 

1 

1, the cubic spline 



/ 3 2 

a\(x— xj) 4-&i(x —xi) +ei(x — xi) +iafi ( 

< x < X 2 


S{x) = { 

a 2 (x - x 2 ) 3 4-b 2 (x - x 2 ) 2 4- c 2 (x - x 2 ) 4- d 2 , 

a„_l (x - x„_0 3 4- b n -i (x - x M _!) 2 




^ 4-c h _i(x —x„_i) 4 d„-i 

7l'v 3 _1 <X<X n 


that interpolates these points has coefficients given by 




a i = (M i +i-M i )!6k 

bi = Mi / 2 

c,- = 0.-+1 -yd lh- [(M 1+ 1 + 2 Mdh / 6] 
di=yi 


(14) 

for i = 1 , 2, ..., n - 1, where M i = S ll {x i ), i = 1 , 2, ..., n. 




From this result, we see that the quantities My •••» M n uniquely determine the cubic spline. To find these quantities, we 
substitute the expressions for a,-, and c,- given in 14 into 12. After some algebraic simplification, we obtain 

M\ 4 4M 2 + M 3 = 6(y\- 2^2 4 ^ 3 ) / h 2 

M 2 4 4M 3 4- M 4 — 6(y 2 - 273 + ^ 4 ) / h 2 


or, in matrix form, 


^n—2 4 4M n —\ 4 M n = 6 (y H _2 — 2y n -\ 4-y n ) f k 2 



Mi 



0 

0 

0 

0 

0 


m 2 


y\-2y 2 + y 3 

0 1 4 1 . 0 0 0 0 


m 3 


yi ~ 2y 3 4 y 4 

0014 - 0000 


m 4 

6 

y3-"4y 4 4-y5 

0 0 0 0 - 4 1 0 0 



h 2 

yn— 4 — 2y n —3 4 y H —2 

0 0 0 0 . 1 4 1 0 


^n—2 


y yi— 3 — 2y yi—2 1 yn —1 

0 0 0 0 - 0 1 4 1 




yn- 2 - 2y„-i 4 y n 






This is a linear system of n _ 2 equations for the n unknowns My M„- Thus, we still need two additional equations to 
determine My My • • •> M n uniquely. The reason for this is that there are infinitely many cubic splines that interpolate the given 
points, so we simply do not have enough conditions to determine a unique cubic spline passing through the points. We discuss 
below three possible ways of specifying the two additional conditions required to obtain a unique cubic spline through the points. 
(The exercises present two more.) They are summarized in Table 1. 


Table 1 











Natural The Second M\ = 0 

Spline derivative of the M —0 

spline is zero at the 
endpoints. 

"4 1 0 - 000 " 

14 1-000 

0 0 0 - 1 4 1 

0 0 0 - 0 1 4 

' M 2 ' 
m 3 

^n—2 

Mft—i 

4 * 

11 

y\ — 2^2 -f 
yi- 273 + ^4 

y»— 2 — 1 fy'M 


Parabolic The spline reduces M\ = M 3 

Runout to a parabolic curve M _ M 1 

Spline on the first and last 

intervals. 

'510-00 O’ 

1 4 1 - 0 0 0 

0 0 0 - 1 4 1 

0 0 0 - 0 1 5 

" m 2 ' 
m 3 

2 

M n —\ 

11 

y\ -Zy 2 + y 2 

y2-2y3 + y4 

yn -2 - tyn- 1 -1 y n 


Cubic The spline is a M\ = 2 M 2 — M 3 

Runout single cubic curve ^ = 2M ^_ { _ M „_ 2 

Spline on the first two and 

last two intervals. 

’6 0 0 - 0 0 0’ 

14 1-000 

0 0 0 - 1 4 1 

0 0 0 - 0 0 6 

' M 2 
m 3 

^yi—2 

Mft—i 

4"* 

II 

y\- 2 y 2 -\-y 2 
y2-2y 3 + y4 

y»—2 2jy M _ 1 1 y n 



The Natural Spline 


The two simplest mathematical conditions we can impose are 

Mi = Af„ = 0 

These conditions together with 15 result in an n x n linear system for M^ M 3 > • which can be written in matrix form as 


"lOOO- 
14 10 - 

■OOO’ 

-000 


’ M\ 
M 2 


0 

y\-2y 2 +y 3 

0 14 1 - 

-000 


m 3 

6 

y2 - 2^3 + y4 

1- 

O O ■■■ 

O O ■■■ 

O O ■■■ 

O O ■■■ 

■14 1 

-001 


M yi — 1 
M n 

~k 2 

yn—2 ~ 2>'m —1 4- yn 

0 


For numerical calculations it is more convenient to eliminate M\ and M n from this system and write 


” 4100 - 
14 10- 

- 0 0 o’ 
-000 


m 2 

m 3 


y\ -272 + 73 

72-2^3+74 

0 14 1- 

0 

0 

0 


m a 

6 

73-2^4 + 75 

0 0 0 0- 

0 0 0 0- 

-14 1 

-014 


^n—2 
Mn — i 

"a 2 

yn —3 2 i y„_2 4 yn— 1 

yn-2 - tyn-1 4 yn 


together with 

ilfl = 0 


(16) 


(17) 


M n = 0 


(18) 


Thus, the (« _ 2) x (« — 2) linear system can be solved for the n — 2 coefficients M 2 ' My •••■> M„_j, and M\ and M n are 
determined by 17 and 18. 


Physically, the natural spline results when the ends of a drafting spline extend freely beyond the interpolating points without 
constraint. The end portions of the spline outside the interpolating points will fall on straight line paths, causing S !! (x) to vanish at 






































the endpoints xj and x n and resulting in the mathematical conditions M i = M H = 0. 


The natural spline tends to flatten the interpolating curve at the endpoints, which may be undesirable. Of course, if it is required 
that S''' (x) vanish at the endpoints, then the natural spline must be used. 


The Parabolic Runout Spline 

The two additional constraints imposed for this type of spline are 

M i = M2 


( 19 ) 


M n — M n —\ 


( 20 ) 


If we use the preceding two equations to eliminate M\ and M n from 15 , we obtain the ( n _ 2) x (n — 2 ) linear system 


L.h 

1—«■ 

O 

0 

- 0 0 o’ 


m 2 


y\ -272 + ys 

14 10- 

-000 


m 3 


72-2^3 bj4 

0 14 1- 

-000 


m 4 

6 

y3-2y 4 +y5 

0 0 0 0 - 

-14 1 


^■n —2 

" h 2 

y»— 3 — 2 ~)~yn— 1 

0 0 0 0 - 

-015 


1 


y»— 2 4- yn 


for My, • • •, M n -\- Once these n — 2 values have been determined, M\ and M n are determined from 19 and 20 . 


( 21 ) 


From 14 we see that M\ = M2 implies that fll = 0, and M n = M„_ 1 implies that a n -\ = 0 - Thus, from 3 there are no cubic terms 
in the formula for the spline over the end intervals [^ 1; ^ 2 ] and ^ * ]. Hence, as the name suggests, the parabolic runout 
spline reduces to a parabolic curve over these end intervals. 

The Cubic Runout Spline 

For this type of spline, we impose the two additional conditions 

Mi = 2M 2 - M 2 no . 


M n — 2 M n —i — M n —2 


( 23 ) 


Using these two equations to eliminate M\ and M n from 15 results in the following (« _ 2 ) x (« — 2 ) linear system for My> My 


•••> M n —\- 


6 

0 

0 

0 - 

- 0 

0 

1 

4 

1 

0 - 

- 0 

0 

0 

1 

4 

1 - 

- 0 

0 

0 

0 

0 

0 - 

- 1 

4 

0 

0 

0 

0 - 

- 0 

0 


1 

m 2 


71 - 2^2+73 


M 2 


72-273 + 74 


m 4 

6 

73 - 274 4 75 



' h 2 



^n—2 


yn— 3 2 i y H _2 4- yn —1 

- 



1 

3 

1 

to 

1 

1_ 


( 24 ) 


After we solve this linear system for My< My • ••■> M n -v w e can use 22 and 23 to determine M\ and M n - 


If we rewrite 22 as 


M 2 — M\ = M^ — M 2 

it follows from 14 that a\=ay Because S !ff (x) = 6 a 1 on [^ 1; X2 ] and S !!, (x) = 6^2 on [x 2 , *3]* we see that S ,ff (x) is constant 
over the entire interval [7-^ 7-7]. Consequently, Six) consists of a single cubic curve over the interval [ X \, xj] rather than two 
different cubic curves pieced together at xy [To see this, integrate S !!! (x) three times.] A similar analysis shows that S(x) consists 
of a single cubic curve over the last two intervals. 














Whereas the natural spline tends to produce an interpolating curve that is flat at the endpoints, the cubic runout spline has the 
opposite tendency: it produces a curve with pronounced curvature at the endpoints. If neither behavior is desired, the parabolic 
runout spline is a reasonable compromise. 


Using a Parabolic Runout Spline 


The density of water is well known to reach a maximum at a temperature slightly above freezing. Table 2, from the Handbook of 
Chemistry and Physics (Cleveland, Ohio: Chemical Rubber Publishing Company), gives the density of water in grams per cubic 
centimeter for five equally spaced temperatures from -10° C to 30° C. We will interpolate these five temperature-density 
measurements with a parabolic runout spline and attempt to find the maximum density of water in this range by finding the 
maximum value on this cubic spline. In the exercises we ask the reader to perform similar calculations using a natural spline and a 
cubic runout spline to interpolate the data points. 


Table 2 


Temperature (° C) 

Density (g/cm^) 

-10 

.99815 

0 

.99987 

10 

.99973 

20 

.99823 

30 

.99567 

= — 10 , 

71 = 99815 

to 

II 

0 

72 = .99987 

LO 

II 

0 

73 = .99973 

0 

II 

’=T 

4 

74 =.99823 

0 

m 

II 

^■"1 

y 5 =99567 


Then 

6[7l-2724 73 ] tk 2 = -.0001116 
6 [ 72 - 2^3 4 74 ] ih 2 = - .0000816 
6 [73-2744 75] ih 2 = - .0000636 
and the linear system 21 for the parabolic runout spline becomes 


'5 1 O' 


~m 2 


'—.000116" 

1 4 1 


m 3 

= 

-.000816 

0 1 5 


m 4 


-.000636 


Solving this system yields 


M 2 = - .00001973 
M 3 = - .00001293 
M 4 = -.00001013 












From 19 and 20, we have 


Mi=M 2 = - .00001973 
M 5 = M 4 = - .00001013 


Solving for the a,-/s, b if s , c,-/s, and ^ 3 -/ s in 14, 
/ 


£(*) = 


.000000113 (x-0) 3 
.000000047 (x- 10) 3 




we obtain the following expression for the interpolating parabolic runout spline: 


- .00000987 (x + 10) 2 4- .0002707 (x 4 10) 4- .99815, 

- .00000987 (x — 0) 2 4- .0000733 (x — 0) 4- .99987, 

- .00000647 (x - 10) 2 - .0000900 (x - 10) 4- .99973, 

- .00000507 (x - 20) 2 - .0002053 (x - 20) 4- .99823, 


— 10 <x < 0 
0<x< 10 


10 <x < 20 
20 <x < 30 


This spline is plotted in Figure 11.5.5. From that figure we see that the maximum is attained in the interval [0, 10]. To find this 
maximum, we set S*(x) equal to zero in the interval [0, 10]: 


S'(x) = .000000339x 2 -.0000197x I .0000733 = 0 


To three significant digits the root of this quadratic in the interval [0, 10] is j7l - = 39 9, and for this value of x, S( 3.99) = 1.00001- 
Thus, according to our interpolated estimate, the maximum density of water is 1.00001 g/cm attained at 3.99° C. This agrees well 

a 

with the experimental maximum density of 1.00000 g/cm attained at 3.98° C. (In the original metric system, the gram was defined 
as the mass of one cubic centimeter of water at its maximum density.) 



♦ 


Closing Remarks 

In addition to producing excellent interpolating curves, cubic splines and their generalizations are useful for numerical integration 
and differentiation, for the numerical solution of differential and integral equations, and in optimization theory. 


Exercise Set 11.5 



Click here for Just Ask! 


Derive the expressions for a 3 - and c 3 in Equations 14 of Theorem 11.5.1. 

1 . 


2. 


The six points 

( 0 , . 00000 ), 


(.2, .19867), (.4, .38942), (.6, .56464), (.8, .71736), (1.0, .84147) 





















lie on the graph of y = sin x, 


where x is in radians. 


(a) Find the portion of the parabolic runout spline that interpolates these six points for .4 < x < .6- Maintain an accuracy of 
five decimal places in your calculations. 

(b) Calculate S(.5") for the spline you found in part (a). What is the percentage error of S(.5) with respect to the “exact” 
value of sin (.5) = 47943? 


The following five points 


lie on a single cubic curve. 


(0, 1), (1, 7), (2, 27) (3, 79), (4, 181) 


(a) Which of the three types of cubic splines (natural, parabolic runout, or cubic runout) would agree exactly with the single 
cubic curve on which the five points lie? 

(b) Determine the cubic spline you chose in part (a), and verify that it is a single cubic curve that interpolates the five 
points. 


Repeat the calculations in Example 1 using a natural spline to interpolate the five data points. 


Repeat the calculations in Example 1 using a cubic runout spline to interpolate the five data points. 


Consider the five points (0, 0), (.5, 1), (1, 0), (1.5, -1), and (2, 0) on the graph of y = sm (ttx)- 

(a) Use a natural spline to interpolate the data points (0, 0), (.5, 1), and (1,0). 

(b) Use a natural spline to interpolate the data points (.5, 1), (1, 0), and (1.5, -1). 

(c) Explain the unusual nature of your result in part (b). 


(The Periodic Spline) If it is known or if it is desired that the n points (x\, yi"), (x 2 , y 2 )’ •••> (x H , y n ) to be interpolated lie 
on a single cycle of a periodic curve with period x n — x\, then an interpolating cubic spline Six) must satisfy 

S(x l )=S(x») 

S*(x l )=S l (x») 

S"(x l )=S ,, (x») 


(a) Show that these three periodicity conditions require that 







y 1 =yn 

Mi = M n 

4Mh M 2 A M„-i = 6(> h _i - 2y\ + y 2 ) th 2 

(b) Using the three equations in part (a) and Equations 15, construct an (« _ ]) x (n — 1) linear system for My M 2 , •••, 
M n - 1 in matrix form. 


8. (The Clamped Spline) Suppose that, in addition to the n points to be interpolated, we are given specific values y'j and y* for 
the slopes S'{x{) and S* {x n ) of the interpolating cubic spline at the endpoints xq and x n . 


(a) Show that 


2Mi + M 2 = 6(y 2 -y \ -hy ! t ) / h 2 
2 M„ + M h _i = 60„_i -y» + hy' n )!h 2 

(b) Using the equations in part (a) and Equations 15, construct an n x n linear system for My My •••• M n in matrix form. 


Remark The clamped spline described in this exercise is the most accurate type of spline for interpolation work if the slopes at 
the endpoints are known or can be estimated. 


Section 11.5 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Tl. 


In the solution of the natural cubic spline problem, it is necessary to solve a system of equations having coefficient matrix 


’4 

1 

0 - 

0 

0 

o’ 

1 

4 

1 - 

0 

0 

0 

0 

0 

0 - 

1 

4 

1 

0 

0 

0 - 

0 

1 

4 


If we can present a formula for the inverse of this matrix, then the solution for the natural cubic spline problem can be easil 
obtained. In this exercise and the next, we use a computer to discover this formula. Toward this end, we first determine an 
expression for the determinant of ^ , denoted by the symbol n,,. Given that 

A\ — [4] and A 2 = 4 ] 


we see that 












T2. 


and 


D i = det (^i) = det [4] =4 


D 2 = det (A 2 ) = det 


4 1 
1 4 


= 15 


(a) Use the cofactor expansion of determinants to show that 

D n = AD n —\ — D n —2 

for n = 3, 4, 5, .... This says, for example, that 

Z) 3 =4D 2 -Z)i =4(15) -4 = 56 
D 4 = 4D 3 -D 2 =4(56) - 15 = 209 

and so on. Using a computer, check this result for 5<«< 10. 


(b) By writing 

D n = AD n —\ — D n —2 

and the identity, 1 =D„_i, in matrix form, 


D n 


'4 

- r 

^n— 1 

1 


_i 

0_ 

2 


show that 


' D» " 


'4 

-T 

h—2 

'D 2 


'A 

- f 

h— 2 

'15' 

l 

7 

£ 

Q 

_ 1 


_i 

0 _ 


d k 


_1 

0 _ 


_ A _ 


(c) Use the methods in Section 7.2 and a computer to show that 


4 

1 



and hence 


for n = \, 2, 3, .... 


(2 + v^r 1 - (2 - /3)”- 1 (2 - f3f~ 2 - (2 + f3f~ 2 
(2 f f3)”~ 2 - (2 - /I)”" 2 (2 - f3)”~ 2 - (2 + ^ 

l{3 

D (2- H /3r i -(2-'/3r i 
2/3 


(d) Using a computer, check this result for 1 < « < 10. 


In this exercise, we determine a formula for calculating ^-1 from for k = 0, 1, 2, 3, n, assuming that £)q is defined t 
be 1. 


(a) Use a computer to compute for £ = ], 2, 3, 4, and 5. 

K 


(b) From your results in part (a), discover the conjecture that 

■An = [ Q Ij] 


where q,j = and 


Q» = (“l) 


i +] I -Qh-j A -1 


for i < 


(c) Use the result in part (b) to compute and compare it to the result obtained using the computer. 
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11.6 

MARKOV CHAINS 


In this section we describe a general model of a system that changes from state 
to state. We then apply the model to several concrete problems. 


Linear Systems 
Matrices 

Intuitive Understanding of Limits 


A Markov Process 

Suppose a physical or mathematical system undergoes a process of change such that at any moment it can occupy one of a finite 
number of states. For example, the weather in a certain city could be in one of three possible states: sunny, cloudy, or rainy. Or an 
individual could be in one of four possible emotional states: happy, sad, angry, or apprehensive. Suppose that such a system 
changes with time from one state to another and at scheduled times the state of the system is observed. If the state of the system at 
any observation cannot be predicted with certainty, but the probability that a given state occurs can be predicted by just knowing 
the state of the system at the preceding observation, then the process of change is called a Markov chain or Markov process. 


DEFINITION 


If a Markov chain has k possible states, which we label as 1, 2, ..., k, then the probability that the system is in state i at any 
observation after it was in state j at the preceding observation is denoted by Pij and is called the transition probability from 
state j to state i. The matrix P = [p i j ] is called the transition matrix of the Markov chain. 


; or example, in a three-state Markov chain, the transition matrix has the form 


Preceding State 


1 

2 

3 


Pu 

m 

Pn 

I 

Pit 

P:.i 

PX 

2 

Pi i 

Pn 

P33j 

3 


New Stale 


In this matrix, p^2 is the probability that the system will change from state 2 to state 3, p\ \ is the probability that the system will 
still be in state 1 if it was previously in state 1, and so forth. 


Transition Matrix of the Markov Chain 


A car rental agency has three rental locations, denoted by 1, 2, and 3. A customer may rent a car from any of the three locations 
and return the car to any of the three locations. The manager finds that customers return the cars to the various locations according 
to the following probabilities: 













Ren Uni from Location 


1 

2 

3 



r.8 .3 

.2 

\ 

Returned 

j 

2 

.6 

2 

m 

to 

.1 ,5 

.2 

3 

LfJCiltLiin 


This matrix is the transition matrix of the system considered as a Markov chain. From this matrix, the probability is .6 that a car 
rented from location 3 will be returned to location 2, the probability is .8 that a car rented from location 1 will be returned to 
location 1, and so forth. 


Transition Matrix of the Markov Chain 


By reviewing its donation records, the alumni office of a college finds that 80% of its alumni who contribute to the annual fund 
one year will also contribute the next year, and 30% of those who do not contribute one year will contribute the next. This can be 
viewed as a Markov chain with two states: state 1 corresponds to an alumnus giving a donation in any one year, and state 2 
corresponds to the alumnus not giving a donation in that year. The transition matrix is 


In the examples above, the transition matrices of the Markov chains have the property that the entries in any column sum to 1. This 
is not accidental. If P = [p i ■] is the transition matrix of any Markov chain with k states, then for each j we must have 

Plj + P2j+- + Pkj=l (1) 

because if the system is in state j at one observation, it is certain to be in one of the k possible states at the next observation. 

A matrix with property 1 is called a stochastic matrix, a probability matrix, or a Markov matrix. From the preceding discussion, it 
follows that the transition matrix for a Markov chain must be a stochastic matrix. 


In a Markov chain, the state of the system at any observation time cannot generally be determined with certainty. The best one can 
usually do is specify probabilities for each of the possible states. For example, in a Markov chain with three states, we might 
describe the possible state of the system at some observation time by a column vector 


*1 


x = 


*2 


x 3 


in which xj is the probability that the system is in state 1, xj the probability that it is in state 2, and xj the probability that it is in 
state 3. In general we make the following definition. 


DEFINITION 


The state vector for an observation of a Markov chain with k states is a column vector x whose ith component x* is the 
probability that the system is in the ith state at that time. 

Observe that the entries in any state vector for a Markov chain are nonnegative and have a sum of 1. (Why?) A column vector that 
has this property is called a probability vector. 


Let us suppose now that we know the state vector x C 1 - 1 ’ 1 for a Markov chain at some initial observation. The following theorem will 
enable us to determine the state vectors 


x d) M) («) 

A j n - - 7 















at the subsequent observation times. 


THEOREM 11.6.1 


If P is the transition matrix of a Markov chain and x (”) is the state vector at the nth observation, then X ( H +1) = p x ( M ). 


The proof of this theorem involves ideas from probability theory and will not be given here. From this theorem, it follows that 

X^Wx^W^ 

x® = J Px^=P 3 x^ 

x (n) = p x (n-r )=p n x (<f 

In this way, the initial state vector X (P) and the transition matrix P determine X ( H ) for « = 1, 2, .... 


Example 2 Revisited 


The transition matrix in Example 2 was 



.3 

.7 


We now construct the probable future donation record of a new graduate who did not give a donation in the initial year after 
graduation. For such a graduate the system is initially in state 2 with certainty, so the initial state vector is 


x 


®> = 


0 

1 


From Theorem 11.6.1 we then have 


x^ 

x® 

x® 


Px® = 

= 

Px® = 


m 

CO 

"O' 


'.3" 


2 ,7_ 

_1_ 


_.7_ 


m 

CO 

'.3' 


'.45' 

2 .7 

.7 


.55 


8 

.3' 

'.45' 


'.525' 

2 

.7 

_.55_ 


_.475_ 


Thus, after three years the alumnus can be expected to make a donation with probability .525. Beyond three years, we find the 
following state vectors (to three decimal places): 


'.563' 

. & = 

'.581' 

. *® = 

'.591' 

. ^ = 

'.595' 

_.438_ 

_ .4 1 9 _ 

_.409_ 

_.405_ 

'.598' 


'.599' 

x m = 

'.599' 

, ^ = 

'.600' 

_.402_ 


_ .401 _ 


_ .401 _ 

_.400_ 


For all n beyond 11, we have 


x (") = 


.600 

.400 


to three decimal places. In other words, the state vectors converge to a fixed vector as the number of observations increases. (We 
shall discuss this further below.) 

♦ 













































EXAM Example 1 Revisited 


The transition matrix in Example 1 was 

“8 .3 .2 

.1 .2 .6 
.1 .5 .2 

If a car is rented initially from location 2, then the initial state vector is 

0 
1 
0 


x© = 


Using this vector and Theorem 11.6.1, one obtains the later state vectors listed in Table 1. 

Table 1 


.(h) 

0 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

(h) 

1 

0 

.300 

.400 

.477 

.511 

.533 

.544 

.550 

.553 

.555 

.556 

.557 

(h) 

2 

1 

.200 

.370 

.252 

.261 

.240 

.238 

.233 

.232 

.231 

.230 

.230 

.(«) 

0 

.500 

.230 

.271 

.228 

.227 

.219 

.217 

.215 

.214 

.214 

.213 


For all values of n greater than 11, all state vectors are equal to to three decimal places. 

Two things should be observed in this example. First, it was not necessary to know how long a customer kept the car. That is, in a 
Markov process the time period between observations need not be regular. Second, the state vectors approach a fixed vector as n 
increases, just as in the first example. 


EXAMPLE 5 Using Theorem 11.6.1 


A traffic officer is assigned to control the traffic at the eight intersections indicated in Figure 11.6.1. She is instructed to remain at 
each intersection for an hour and then to either remain at the same intersection or move to a neighboring intersection. To avoid 
establishing a pattern, she is told to choose her new intersection on a random basis, with each possible choice equally likely. For 
example, if she is at intersection 5, her next intersection can be 2, 4, 5, or 8, each with probability Every day she starts at the 
location where she stopped the day before. The transition matrix for this Markov chain is 












Ok! Intersection 
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0 

0 

I 

0 

4 

4 

i 

4 


H 

0 

0 
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8 
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Intersection 



n f ii i r 

Figure 11.6.1 

If the traffic officer begins at intersection 5, her probable locations, hour by hour, are given by the state vectors given in Table 2. 
For all values of n greater than 22, all state vectors are equal to x ■ to three decimal places. Thus, as with the first two examples, 
the state vectors approach a fixed vector as n increases. 

Table 2 








n 





*(») 

0 

1 

2 

3 

4 

5 

10 

15 

20 

22 

(«) 

*1 

0 

.000 

.133 

.116 

.130 

.123 

.113 

.109 

.108 

.107 

(n) 

x 2 

0 

.250 

.146 

.163 

.140 

.138 

.115 

.109 

.108 

.107 

(h) 

*3 

0 

.000 

.050 

.039 

.067 

.073 

.100 

.106 

.107 

.107 

(n) 

*4 

0 

.250 

.113 

.187 

.162 

.178 

.178 

.179 

.179 

.179 

(«) 

x 5 

1 

.250 

.279 

.190 

.190 

.168 

.149 

.144 

.143 

.143 

_(«) 

x 6 

0 

.000 

.000 

.050 

.056 

.074 

.099 

.105 

.107 

.107 































n 






0 

1 

2 

3 

4 

5 

10 

15 

20 

22 

(h) 

x 7 

0 

.000 

.133 

.104 

.131 

.125 

.138 

.142 

.143 

.143 

(«) 

*8 

0 

.250 

.146 

.152 

.124 

.121 

.108 

.107 

.107 

.107 


♦ 


Limiting Behavior of the State Vectors 

In our examples we saw that the state vectors approached some fixed vector as the number of observations increased. We now ask 
whether the state vectors always approach a fixed vector in a Markov chain. A simple example shows that this is not the case. 


System Oscillates between Two State Vectors 


Let 

f 

0 _ 

Then, because p' 1 = i and p-' _ p, we have that 

x®=*®=*«>=...=iy 

_o_ 

and 

x®=x®=*® = •■•=[ 5’ 

, so it does not approach any fixed vector. 

♦ 

However, if we impose a mild condition on the transition matrix, we can show that a fixed limiting state vector is approached. This 
condition is described by the following definition. 


This system oscillates indefinitely between the two state vectors 


and 


P = 


0 1 
1 0 


and 


x® = 


DEFINITION 


A transition matrix is regular if some integer power of it has all positive entries. 


Thus, for a regular transition matrix P, there is some positive integer m such that all entries of p m are positive. This is the case with 
the transition matrices of Examples Example 1 and Example 2 for m = \. In Example 5 it turns out that has all positive entries. 
Consequently, in all three examples the transition matrices are regular. 

A Markov chain that is governed by a regular transition matrix is called a regular Markov chain. We shall see that every regular 
Markov chain has a fixed state vector q such that p n x^ approaches q as n increases for any choice of x ®- This result is of major 
importance in the theory of Markov chains. It is based on the following theorem. 


























THEOREM 11.6.2 


Behavior of p" as n -*■ » 




If P is a regular transition matrix , then as n _ * oo , 





'01 

01 

- 01' 


p n -» 

02 

02 

... q2 



0ft 

0ft 

- 0ft 


where the ■?,- are positive numbers such that qi \ 

1 02 

+ ■■■- 

1 0ft = 

1. 


We will not prove this theorem here. The interested reader is referred to a more specialized text, such as J. Kemeny and J. Snell, 
Finite Markov Chains (New York: Springer-Verlag, 1976). 


Let us set 


'01 

01 • 

'• 01' 



' 01 ' 

02 

02 ■ 

- 02 

and 

q = 

02 

0ft 

0ft ‘ 

'• 0ft 



0ft 


Thus, Q is a transition matrix, all of whose columns are equal to the probability vector q. Q has the property that if x is any 
probability vector, then 



'01 01 ■ 
02 02 ' 

1- 

-h r-i 
^ ^ 

'^r 

x 2 

— 

'01X1 +<3 , ix 2 + - 
02*1 + 02*2+ ■' 

'■+ 01*ft’ 
'■+ 02*ft 


1 

'■ 0ft 



0ft*l + 0ft*2 + 

■ + 0ft*ft 


= (*l 


*2 H-b*ft) 


01 

02 


= (l)q = q 


0^ 


That is, Q transforms any probability vector x into the fixed probability vector q. This result leads to the following theorem. 


THEOREM 11.6.3 


Behavior of P n x as n x 



IfP is a regular transition matrix and x is any probability vector , then as n _ + oo > 

P n x -+ 

' 01 ' 

02 

= q 


0ft 


where q is a fixed probability vector, independent of n, 

all of whose entries are positive. 


This result holds since Theorem 11.6.2 implies that P n - * Q as n _ * oo • This in turn implies that P n x - * Qx = q as 

n —* rxj. Thus, for a regular Markov chain, the system eventually approaches a fixed state vector q. The vector q is called the 
steady-state vector of the regular Markov chain. 

For systems with many states, usually the most efficient technique of computing the steady-state vector q is simply to calculate P n i 
for some large n. Our examples illustrate this procedure. Each is a regular Markov process, so that convergence to a steady-state 
vector is ensured. Another way of computing the steady-state vector is to make use of the following theorem. 

























THEOREM 11.6.4 


Steady-State Vector 

The steady-state vector q of a regular transition matrix P is the unique probability vector that satisfies the equation = q. 


To see this, consider the matrix identity ppn _ By Theorem 11.6.2, both p n and pn +1 approach Q as n —> qq . Thus, we 

have PQ — Q. Any one column of this matrix equation gives Pq = q. To show that q is the only probability vector that satisfies this 
equation, suppose r is another probability vector such that Pr = i. Then also P K r = r for « = 1, 2, .... When we let n —> oo, 
Theorem 11.6.3 leads to q = r. 

Theorem 11.6.4 can also be expressed by the statement that the homogeneous linear system 

(/ —P)q = 0 

has a unique solution vector q with nonnegative entries that satisfy the condition q j | ^4- I q^ = \- We can apply this 
technique to the computation of the steady-state vectors for our examples. 


Example 2 Revisited 


In Example 2 the transition matrix was 


so the linear system (/ _ p) q = o is 


This leads to the single independent equation 


or 


P = 


.8 .3 
.2 .7 


.2 -.3' 

'0f 


'O' 

-1 

i:- - i 

i 

_i 

02 


_o_ 


( 2 ) 


■2?i - .3?2 = 0 
fl - 1-502 


Thus, when we set $2 = s, any solution of 2 is of the form 


q = S 


1.5 

1 


where s is an arbitrary constant. To make the vector q a probability vector, we set s = 1 / (1.5 4- 1) = .4- Consequently, 

’. 6 ' 


q = 


.4 


is the steady-state vector of this regular Markov chain. This means that over the long run, 60% of the alumni will give a donation 
in any one year, and 40% will not. Observe that this agrees with the result obtained numerically in Example 3. 


Example 1 Revisited 


In Example 1 the transition matrix was 





















so the linear system (/ — P)q = 0 is 


.2 

-.1 

-.3 

.8 

1 - 

C"-] '-O 

1 1 

02 

_ 

1 - 

o o 

1 _ 

-.1 

-.5 

.8 

03 


0 


The reduced row-echelon form of the coefficient matrix is (verify) 


1 0 

0 1 
0 0 


34 

13 

14 
13 

0 


so the original linear system is equivalent to the system 



When we set #3 = s, any solution of the linear system is of the form 


q = £ 


34 

13 

14 
13 
1 


To make this a probability vector, we set 


Thus, the steady-state vector of the system is 


s = ■ 


1 


31 h 

13 


M + i 

13 


II 

61 


<1 = 


34 

61 

14 

61 

13 

61 


.5573... 

.2295... 

.2131... 


This agrees with the result obtained numerically in Table 1. The entries of q give the long-run probabilities that any one car will be 
returned to location 1, 2, or 3, respectively. If the car rental agency has a fleet of 1000 cars, it should design its facilities so that 
there are at least 558 spaces at location 1, at least 230 spaces at location 2, and at least 214 spaces at location 3. 

♦ 


Example 5 Revisited 


We will not give the details of the calculations but simply state that the unique probability vector solution of the linear system 
(/ — P) q = 0 is 





















3 _ 

28 

_3_ 

28 

_3_ 

28 

_5_ 

28 

4 _ 

28 

_3_ 

28 

4 _ 

28 

3 _ 

28 


1071.. 

1071.. 

1071.. 

1785.. 

1428.. 

1071.. 

1428.. 

1071.. 


The entries in this vector indicate the proportion of time the traffic officer spends at each intersection over the long term. Thus, if 
the objective is for her to spend the same proportion of time at each intersection, then the strategy of random movement with equal 
probabilities from one intersection to another is not a good one. (See Exercise 5.) 


Exercise Set 11.6 



Click here for Just Ask! 


1 . 


Consider the transition matrix 



.5 

.5 


W Calculate 


(b) State why P is regular and find its steady-state vector. 


2 . 


Consider the transition matrix 


P = 


.2 

.6 

.2 


.1 

.4 

.5 


.7 

.2 

.1 


(a) Calculate *0), x (9, and ^C 3 ) to three decimal places if 


x© = 


0 

0 

1 


(b) State why P is regular and find its steady-state vector. 

















Find the steady-state vectors of the following regular transition matrices: 


3. 


(a) 


1 1 

3 4 

2 1 

3 4 


(b) 


.81 .26 
.19 .74 


(c) 


1 1 0 
3 2 


i 0 

i i 
3 2 


Let P be the transition matrix 

4. 


1 

2 

1 

2 


0 

1 


(a) Show that P is not regular. 


(b) 


Show that as n increases, p n \^ approaches 


0 

1 


for any initial state vector 


(c) What conclusion of Theorem 11.6.3 is not valid for the steady state of this transition matrix? 


Verify that if P is a fc x k regular transition matrix all of whose row sums are equal to 1, then the entries of its steady-state 
5- vector are all equal to ] / 1 : - 


Show that the transition matrix 


6 . 


P = 


0 TT 


1 

2 

1 

2 

■k 0 


1 

2 

0 

1 

2 


is regular, and use Exercise 5 to find its steady-state vector. 


7. 


John is either happy or sad. If he is happy one day, then he is happy the next day four times out of five. If he is sad one day, 
then he is sad the next day one time out of three. Over the long term, what are the chances that John is happy on any given day? 


A country is divided into three demographic regions. It is found that each year 5% of the residents of region 1 move to region 
8- and 5% move to region 3. Of the residents of region 2, 15% move to region 1 and 10% move to region 3. And of the residents 




of region 3, 10% move to region 1 and 5% move to region 2. What percentage of the population resides in each of the three 
regions after a long period of time? 


Section 11.6 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Consider the sequence of transition matrices 

Tl. 


with 


{P 2 , P 3 . Pa,-) 


Pi = 


0 ^ 


1 ^ 


0 0 



1 

2 

1 

2 


1 

3 

1 

3 

1 

3 


0 


0 


Pa = 


0 


1 


0 0 


1 1 

2 3 

1 1 

2 3 


1 

4 

1 

4 

1 

4 

1 

4 


0 0 
0 0 


P5 = 


0 


0 


1 


0 

1 

2 

1 

2 


0 0 


1 1 

3 4 

1 1 

3 4 

1 1 

3 4 


1 

5 

I 

5 

1 

5 

I 

5 

I 

5 


and so on. 


(a) Use a computer to show that each of these four matrices is regular by computing their squares. 

(b) Verify Theorem 11.6.2 by computing the 100th power of p^ for k = 2, 3, 4, 5. Then make a conjecture as to the 
limiting value of p” as n —* oo for all £ = 2, 3, 4, .... 

(c) Verify that the common column q^ of the limiting matrix you found in part (b) satisfies the equation p^ q fc = q^, as 
required by Theorem 11.6.4. 


T2. 


A mouse is placed in a box with nine rooms as shown in the accompanying figure. Assume that it is equally likely that the 
mouse goes through any door in the room or stays in the room. 











(a) Construct the 9 x 9 transition matrix for this problem and show that it is regular. 

(b) Determine the steady-state vector for the matrix. 

(c) Use a symmetry argument to show that this problem may be solved using only a 3 x 3 matrix. 
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11.7 

GRAPH THEORY 


In this section we introduce matrix representations of relations among members 
of a set. We use matrix arithmetic to analyze these relationships. 


Matrix Addition and Multiplication 


Relations among Members of a Set 

There are countless examples of sets with finitely many members in which some relation exists among members of the set. For 
example, the set could consist of a collection of people, animals, countries, companies, sports teams, or cities; and the relation 
between two members, A and B, of such a set could be that person A dominates person B, animal A feeds on animal B, country A 
militarily supports country B, company A sells its product to company B, sports team A consistently beats sports team B, or city A 
has a direct airline flight to city B. 

We shall now show how the theory of directed graphs can be used to mathematically model relations such as those in the 
preceding examples. 

Directed Graphs 

A directed graph is a finite set of elements, { p 2i ...,P n ) together with a finite collection of ordered pairs (P u Pj) of distinct 
elements of this set, with no ordered pair being repeated. The elements of the set are called vertices, and the ordered pairs are 
called directed edges, of the directed graph. We use the notation P i —> pj (which is read “p. is connected to P”) to indicate that 
the directed edge (p u pj) belongs to the directed graph. Geometrically, we can visualize a directed graph (Figure 11.7.1) by 

representing the vertices as points in the plane and representing the directed edge P i -> Pj by drawing a line or arc from vertex 

p i to vertex Pj, with an arrow pointing from p. to Pj. If both P\- —* pj and Pj —> p i hold (denoted P i _> pj), we draw a single 
line between p i and Pj with two oppositely pointing arrows (as with p 2 and p 2 in the figure). 



Figure 11.7.1 

As in Figure 11.7.1, for example, a directed graph may have separate “components” of vertices that are connected only among 

themselves; and some vertices, such as Py may not be connected with any other vertex. Also, because p._, p i is not permitted 

in a directed graph, a vertex cannot be connected with itself by a single arc that does not pass through any other vertex. 

Figure 11.7.2 shows diagrams representing three more examples of directed graphs. With a directed graph having n vertices, we 
may associate an n x n matrix M = [m^], called the vertex matrix of the directed graph. Its elements are defined by 









Pi * - 


p( 


Pi 


v/*. 


(«) 



<*> 



ft) 


Figure 11.7.2 


1. ffP,- Pj 

0 otherwise 


for i, j = 1,2,... «. For the three directed graphs in Figure 11.7.2, the corresponding vertex matrices are 


Figure 11.7.2a: 


M = 


1 0 0 
0 1 0 
1 0 1 
0 0 0 


Figure 11.7.2&: 


M = 


1 

0 

0 

1 

0 

1 

1 

0 

0 

0 

1 

0 

1 

0 

0 

1 

1 

1 

0 

0 


Figure 11.7.2c: 


M = 


1 0 
0 1 
0 0 
0 0 


0 

0 

1 

0 


By their definition, vertex matrices have the following two properties: 


(i) All entries are either 0 or 1. 


(ii) All diagonal entries are 0. 













Conversely, any matrix with these two properties determines a unique directed graph having the given matrix as its vertex matrix. 
For example, the matrix 


determines the directed graph in Figure 11.7.3. 


M = 


0 1 
0 0 
1 0 
0 0 


1 0 
1 0 
0 1 
0 0 



Figure 11.7.3 


Influences within a Family 


A certain family consists of a mother, father, daughter, and two sons. The family members have influence, or power, over each 
other in the following ways: the mother can influence the daughter and the oldest son; the father can influence the two sons; the 
daughter can influence the father; the oldest son can influence the youngest son; and the youngest son can influence the mother. 
We may model this family influence pattern with a directed graph whose vertices are the five family members. If family member A 
influences family member B, we write A —> B■ Figure 11.7.4 is the resulting directed graph, where we have used obvious letter 
designations for the five family members. The vertex matrix of this directed graph is 


F 

D 

OS 


0 

0 

0 

0 


YS t 


0 

n 

i 

0 

0 


D OS YS 

I 1 if 

0 I 1 

0 0 f) 

0 0 1 

0 0 f) 


M YS 



Figure 11.7.4 


Vertex Matrix: Moves on a Chessboard 


In chess the knight moves in an “L”-shaped pattern about the chessboard. For the board in Figure 11.7.5 it may move horizontally 
two squares and then vertically one square, or it may move vertically two squares and then horizontally one square. Thus, from the 
center square in the figure, the knight may move to any of the eight marked shaded squares. Suppose that the knight is restricted to 












the nine numbered squares in Figure 11.7.6. If by i —> j we mean that the knight may move from square i to square j, the directed 
graph in Figure 11.7.7 illustrates all possible moves that the knight may make among these nine squares. In Figure 11.7.8 we have 
“unraveled” Figure 11.7.7 to make the pattern of possible moves clearer. 
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Figure 11.7.5 
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Figure 11.7.6 

2 3 



Figure 11.7.7 

8 



Figure 11.7.8 


The vertex matrix of this directed graph is given by 






















M = 


0 0 0 0 0 1 0 1 0 

0 0 0 0 0 0 1 0 1 

0 0 0 1 0 0 0 1 0 

0 0 1 0 0 0 0 0 1 

000000000 
1 0 0 0 0 0 1 0 0 

0 1 0 0 0 1 0 0 0 

1 0 1 0 0 0 0 0 0 

0 1 0 1 0 0 0 0 0 

♦ 

In Example 1 the father cannot directly influence the mother; that is, p -> M is not true. But he can influence the youngest son, 

who can then influence the mother. We write this as p —> YS —* M and call it a 2-step connection from F to M. Analogously, 
we call M —> L a 1-step connection, p —► OS —> YS —> M a 3-step connection, and so forth. Let us now consider a 
technique for finding the number of all possible r-step connections (r = 1 , 2 ,...) from one vertex p i to another vertex Pj of an 
arbitrary directed graph. (This will include the case when f\. and Pj are the same vertex.) The number of 1-step connections from 
Pj to Pj is simply That is, there is either zero or one 1-step connection from p. to p,, depending on whether is zero or 

one. For the number of 2-step connections, we consider the square of the vertex matrix. If we let m ® be the (j r y)-th element of 
M 2 , we have 


fKjj = 4- W3j2WJ2j H-b m rn m nj (1) 

Now, if =m\j= 1, there is a 2-step connection Pj -> P^ —> Pj from to Pj. But if either wj/i or m\j is zero, such a 2-step 

connection is not possible. Thus p t —* p^ —> pj is a 2-step connection if and only if = 1. Similarly, for any k = 1, 2, 

...,n,P i ^P k ^P i is a 2 -step connection from to Pj if and only if the term on the right side of 1 is one; 

otherwise, the term is zero. Thus, the right side of 1 is the total number of two 2-step connections from p\- to Pj. 

A similar argument will work for finding the number of 3-, 4-, ..., n-step connections from p. to p,. In general, we have the 
following result. 


THEOREM 11.7.1 


Let M be the vertex matrix of a directed graph and let be the (j ; j)-th element of M r - Then is equal to the number of 
r-step connections from p. to p,. 


Using Theorem 11.7.1 


Figure 11.7.9 is the route map of a small airline that services the four cities Pp p 2 , P 2 , P^. As a directed graph, its vertex matrix is 


M = 


0 110 
10 10 
10 0 1 
0 110 


We have that 
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Figure 11.7.9 


If we are interested in connections from city p 4 to city Py we may use Theorem 11.7.1 to find their number. Because = 
there is one 1-step connection; because _ j, there is one 2-step connection; and because _ 3 , there are three 3-step 


connections. To verify this, from Figure 11.7.9 we find 

1 - step connections from P 4 to P 3 : 

Pa 

P 3 


2 - step connections from P 4 to P 3 : 

Pa —► 

Pi^ 

P 3 

3 - step connections from P 4 to Py. 

Pa —* 


Pa —> P 3 


1 , 


Pa - * Pj -* ^ 1 —> ?! 

Pa - P3 - * 


Cliques 

In everyday language a “clique” is a closely knit group of people (usually three or more) that tends to communicate within itself 
and has no place for outsiders. In graph theory this concept is given a more precise meaning. 


DEFINITION 


A subset of a directed graph is called a clique if it satisfies the following three conditions: 

(i) The subset contains at least three vertices. 

(ii) For each pair of vertices p i and Pj in the subset, both P i —> Pj and Pj —> P i are true. 

(iii) The subset is as large as possible; that is, it is not possible to add another vertex to the subset and still satisfy condition 

(ii). 


This definition suggests that cliques are maximal subsets that are in perfect “communication” with each other. For example, if the 
vertices represent cities, and P i —> Pj means that there is a direct airline flight from city p i to city Pj, then there is a direct flight 
between any two cities within a clique in either direction. 

















XA A Directed Graph with Two Cliques 


The directed graph illustrated in Figure 11.7.10 (which might represent the route map of an airline) has two cliques: 

{P U P 2 ,P 3 ,P A } and {P 3 .P 4 .P 6 } 

This example shows that a directed graph may contain several cliques and that a vertex may simultaneously belong to more than 
one clique. 



Figure 11.7.10 


For simple directed graphs, cliques can be found by inspection. But for large directed graphs, it would be desirable to have a 
systematic procedure for detecting cliques. For this purpose, it will be helpful to define a matrix S = [s^- ] related to a given 
directed graph as follows: 

(1, if Pj+*Pj 

S]i — I 

0, otherwise 

The matrix S determines a directed graph that is the same as the given directed graph, with the exception that the directed edges 
with only one arrow are deleted. For example, if the original directed graph is given by Figure 11.7.11a, the directed graph that has 
S as its vertex matrix is given in Figure 11.7.1 \b. The matrix S may be obtained from the vertex matrix M of the original directed 
graph by setting = 1 if = 1 and setting = 0 otherwise. 






m 

>3 

</>) 









Figure 11.7.11 


The following theorem, which uses the matrix S, is helpful for identifying cliques. 

THEOREM 11.7.2 


Identifying Cliques 


Let c-® be the (j r j)-th element ofsThen a vertex p. belongs to some clique if and only if J?) g. 


Proof If J?) then there is at least one 3-step connection from p i to itself in the modified directed graph determined by S. 
Suppose it is P i —> Pj —> P^ —> Py In the modified directed graph, all directed relations are two-way, so we also have the 
connections P i ^ , Pj P% « P r But this means that [P u Pf Pf) is either a clique or a subset of a clique. In either case, p i must 

belong to some clique. The converse statement, “if p i belongs to a clique, then ^ q,” follows in a similar manner. 


Using Theorem 11.7.2 


Suppose that a directed graph has as its vertex matrix 


0 1 
1 0 
0 1 
1 0 


1 1 
1 0 
0 1 
0 0 


Then 


'o 

1 

0 

f 


'o 

3 

0 

2' 

1 

0 

1 

0 

and S~~' = 

3 

0 

2 

0 

0 

1 

0 

0 

0 

2 

0 

1 

1 

0 

0 

0 


2 

0 

1 

0 


Because all diagonal entries of are zero, it follows from Theorem 11.7.2 that the directed graph has no cliques. 


♦ 


Using Theorem 11.7.2 


Suppose that a directed graph has as its vertex matrix 


M = 


0 10 11 
10 0 10 
110 10 
110 0 0 
10 0 10 


Then 

















s= 


0 
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0 

1 
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0 
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3 
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0 
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3 

1 

0 

0 

0 
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0 

II 

■’"1 

^! 

T ? 
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0 

0 

0 

0 
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1 

0 

0 
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3 

0 

2 

1 

1 

0 

0 

0 

0 


3 

1 

0 

1 

0 


The nonzero diagonal entries of £ 3 are ^C 3 ), S C9, and s &. Consequently, in the given directed graph, p^, p^, and p^ belong to 
cliques. Because a clique must contain at least three vertices, the directed graph has only one clique, { p^ p-^ P ^). 


Dominance-Directed Graphs 

In many groups of individuals or animals, there is a definite “pecking order” or dominance relation between any two members of 
the group. That is, given any two individuals A and B, either A dominates B or B dominates A, but not both. In terms of a directed 

graph in which P i -> Pj means p i dominates Pj, this means that for all distinct pairs, either P i —> Pj or Pj —> Pj, but not 

both. In general, we have the following definition. 


DEFINITION 


A dominance-directed graph is a directed graph such that for any distinct pair of vertices p i and Pj, either P i —> Pj or 
Pj —> Pj, but not both. 


An example of a directed graph satisfying this definition is a league of n sports teams that play each other exactly one time, as in 
one round of a round-robin tournament in which no ties are allowed. If p, —* pj means that team Pj beat team Pj in their single 
match, it is easy to see that the definition of a dominance-directed group is satisfied. For this reason, dominance-directed graphs 
are sometimes called tournaments. 

Figure 11.7.12 illustrates some dominance-directed graphs with three, four, and five vertices, respectively. In these three graphs, 
the circled vertices have the following interesting property: from each one there is either a 1-step or a 2-step connection to any 
other vertex in its graph. In a sports tournament, these vertices would correspond to the most “powerful” teams in the sense that 
these teams either beat any given team or beat some other team that beat the given team. We can now state and prove a theorem 
that guarantees that any dominance-directed graph has at least one vertex with this property. 

















Figure 11.7.12 


THEOREM 11.7.3 


Connections in Dominance-Directed Graphs 

In any dominance-directed graph, there is at least one vertex from which there is a 1-step or 2-step connection to any other 
vertex. 


Proof Consider a vertex (there may be several) with the largest total number of 1-step and 2-step connections to other vertices in 
the graph. By renumbering the vertices, we may assume that p^ is such a vertex. Suppose there is some vertex p. such that there is 

no 1-step or 2-step connection from p j to p^ Then, in particular, p t _> p i is not true, so that by definition of a 

dominance-directed graph, it must be that p i _» Pj. Next, let p k be any vertex such that Pj _, p k is true. Then we cannot have 

p k _» p., as then Pj_, p k _, p i would be a 2-step connection from Pj to p r Thus, it must be that p i _, p k . That is, p i 

has 1-step connections to all the vertices to which Pj has 1-step connections. The vertex p. must then also have 2-step connections 

to all the vertices to which Pj has 2-step connections. But because, in addition, we have that p._, p 1? this means that p. has 

more 1-step and 2-step connections to other vertices than does Pj. However, this contradicts the way in which Pj was chosen. 
Hence, there can be no vertex p. to which Pj has no 1-step or 2-step connection. 


This proof shows that a vertex with the largest total number of 1-step and 2-step connections to other vertices has the property 














stated in the theorem. There is a simple way of finding such vertices using the vertex matrix M and its square M * l - The sum of the 
entries in the zth row of M is the total number of 1-step connections from p i to other vertices, and the sum of the entries of the zth 
row of m 2 is the total number of 2-step connections from p i to other vertices. Consequently, the sum of the entries of the zth row 
of the matrix A = M -V M 2 is the total number of 1-step and 2-step connections from p. to other vertices. In other words, a row of 
A = M f M 2 with the largest row sum identifies a vertex having the property stated in Theorem 11.7.3. 


Using Theorem 11.7.3 


Suppose that five baseball teams play each other exactly once, and the results are as indicated in the dominance-directed graph of 
Figure 11.7.13. The vertex matrix of the graph is 


M = 


0 0 
1 0 
0 0 
0 1 
1 0 


1 1 0 
1 0 1 
0 1 0 
0 0 0 

1 1 0 



Figure 11.7.13 


0 110 
0 10 1 
0 0 10 1 
10 0 0 
0 110 

The row sums of A are 
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1 
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0 
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connection to any other vertex. This is 


1 st row sum = 4 
2nd row sum = 9 
3rd row sum = 2 
4th row sum = 4 
5th row sum = 7 

Because the second row has the largest row sum, the vertex p 2 must have a 1-step or 2-step 
easily verified from Figure 11.7.13. 


so 


A = M+ M a = 


We have informally suggested that a vertex with the largest number of 1-step and 2-step connections to other vertices is a 
“powerful” vertex. We can formalize this concept with the following definition. 


DEFINITION 

















The power of a vertex of a dominance-directed graph is the total number of 1-step and 2-step connections from it to other 
vertices. Alternatively, the power of a vertex p i is the sum of the entries of the zth row of the matrix A = M \ M.' 1 , where M is 
the vertex matrix of the directed graph. 


Example 7 Revisited 


Let us rank the five baseball teams in Example 7 according to their powers. From the calculations for the row sums in that 
example, we have 

Power of team Pi =4 
Power of team P 2 = 9 
Power of team P 3 = 2 
Power ofteamP 4 = 4 
Power of team = 7 

Hence, the ranking of the teams according to their powers would be 

P7 (first), P*, (second), Pi andP4 (tied for third), P3 (last) 

♦ 


Exercise Set 11.7 



Click here for Just Ask! 


1 . 


Construct the vertex matrix for each of the directed graphs illustrated in the accompanying figure. 








0 . 1 ) 






Figure Ex-1 


Draw a diagram of the directed graph corresponding to each of the following vertex matrices. 

2 . 


(a) 


0 110 
10 0 0 
0 0 0 1 
10 10 


(b) 


0 0 10 0 
1 0 0 0 1 
0 10 11 
0 0 0 0 0 
1110 0 


(C) 


0 10 10 1 
1 0 0 0 1 0 
0 0 0 0 0 0 
110 0 10 
0 0 0 1 0 1 
0 10 0 10 













Let M be the following vertex matrix of a directed graph: 


0 111 
10 0 0 
0 10 1 
0 110 


(a) Draw a diagram of the directed graph. 

(b) Use Theorem 11.7.1 to find the number of 1-, 2-, and 3-step connections from the vertex p^ to the vertex p^ Verify 
your answer by listing the various connections as in Example 3. 

(c) Repeat part (b) for the 1-, 2-, and 3-step connections from p^ to p 4 . 


(a) Compute the matrix product m ^M for the vertex matrix M in Example 1. 

(b) Verify that the Mi diagonal entry of M T M the number of family members who influence the Mi family member. Why 
is this true? 

(c) Find a similar interpretation for the values of the non diagonal entries of M 7 M- 


By inspection, locate all cliques in each of the directed graphs illustrated in the accompanying figure. 

5. 



Figure Ex-5 


For each of the following vertex matrices, use Theorem 11.7.2 to find all cliques in the corresponding directed graphs. 

6 . 


0 10 10 
10 10 1 
0 10 11 
1 0 0 0 1 
10 110 


(a) 












(b) 


0 10 110 
10 10 11 
0 10 10 1 
10 10 11 
0 10 10 0 
0 0 1110 


For the dominance-directed graph illustrated in the accompanying figure, construct the vertex matrix and find the power of each 
7. vertex. 



Figure Ex-7 


Five baseball teams play each other one time with the following results: 

A beats B, C, D 
B beats C, E 
Cheats D, E 
D beats B 
Cheats A, D 

Rank the five baseball teams in accordance with the powers of the vertices they correspond to in the dominance-directed graph 
representing the outcomes of the games. 


Section 11.7 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


A graph having n vertices such that every vertex is connected to every other vertex has vertex matrix given by 


0 

1 

1 

1 

1 ■ 

■ l" 

1 

0 

1 

1 

1 ■ 

- 1 

1 

1 

0 

1 

1 ■ 

- 1 

1 

1 

1 

0 

1 ■ 

- 1 

1 

1 

1 

1 

0 ■ 

■ 1 

1 

1 

1 

1 

1 ■ 

- 0 


Tl. 













In this problem we develop a formula for j^k whose j)-th entry equals the number of k-step connections from p i to Pj. 


T2. 


(a) Use a computer to compute the eight matrices j^k for n = 2 , 3 and for £• = 2 , 3, 4, 5. 


(b) Use the results in part (a) and symmetry arguments to show that j^k can be written as 


0 

1 

1 

1 

1 

f 
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Pk 

Pk 

Pk - 

Pk 

1 

0 

1 

1 
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1 

0 

1 

1 
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Pk 

Pk 
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Pk 

Pk - 

Pk 

1 

1 

1 

0 

1 

1 

= 

Pk 

Pk 

Pk 

®k 
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1 

1 

1 

1 
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fh 

Pk 

Pk 

Pk 

Q k - 

Pk 

1 

1 

1 

1 

1 ... 
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3 k 

:3k 

Pk 

Pk 

Pk 

Q k 


(c) Using the fact that j^k _ , show that 


with 


r a *i 


'0 

n — 1 

"o<fc-r 

1 — 


_i 

n — 2 

Pk-1 


‘erf 


'O' 

P 1. 


_1_ 


(d) Using part (c), show that 


~&k 


0 n — 1 

k—1 

'o' 

Pk_ 


_1 »-2_ 


_i_ 


(e) Use the methods of Section 7.2 to compute 


0 n — 1 
1 n — 2 


-ifc—1 


and thereby obtain expressions for a k and 3 k , and eventually show that 




U» I (-1)% 


where jj n is the n x n matrix all of whose entries are ones and is the n x n identity matrix. 


(f) Show that for n > 2, all vertices for these directed graphs belong to cliques. 


Consider a round-robin tournament among n players (labeled a\, ^ 3 , •••> where a\ beats ct 2 , a 2 beats < 23 , beats a 4 , 

a„~\ beats a n , and a n beats a\. Compute the “power” of each player, showing that they all have the same power; then 
determine that common power. 

Hint Use a computer to study the cases n = 3, 4, 5, 6 ; then make a conjecture and prove your conjecture to be true. 
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11.8 

GAMES OF STRATEGY 


In this section we discuss a general game in which two competing players 
choose separate strategies to reach opposing objectives. The optimal strategy 
of each player is found in certain cases with the use of matrix techniques. 


Matrix Multiplication 
Basic Probability Concepts 


Game Theory 


To introduce the basic concepts in the theory of games, we will consider the following carnival-type game that two people agree to 
play. We will call the participants in the game player R and player C. Each player has a stationary wheel with a movable pointer on 
it as in Figure 11.8.1. For reasons that will become clear, we will call player R's wheel the row-wheel and player Cs wheel the 
column-wheel. The row-wheel is divided into three sectors numbered 1, 2, and 3, and the column-wheel is divided into four sectors 
numbered 1, 2, 3, and 4. The fractions of the area occupied by the various sectors are indicated in the figure. To play the game, 
each player spins the pointer of his or her wheel and lets it come to rest at random. The number of the sector in which each pointer 
comes to rest is called the move of that player. Thus, player R has three possible moves and player C has four possible moves. 
Depending on the move each player makes, player C then makes a payment of money to player R according to Table 1. 




of player C 

Figure 11.8.1 


Table 1 

Payment to Player R 


Player C's Move 

1 

2 

3 

4 




















Player C's Move 



1 

2 

3 

4 


1 

$3 

$5 

-$2 

-$1 

Player R's Move 

2 

-$2 

$4 

-$3 

-$4 


3 

$6 

-$5 

$0 

$3 


For example, if the row-wheel pointer comes to rest in sector 1 (player/? makes move 1), and the column-wheel pointer comes to 
rest in sector 2 (player C makes move 2), then player C must pay player R the sum of $5. Some of the entries in this table are 
negative, indicating that player C makes a negative payment to player R. By this we mean that player R makes a positive payment 
to player C. For example, if the row-wheel shows 2 and the column-wheel shows 4, then player R pays player C the sum of $4, 
because the corresponding entry in the table is _ £ 4 . In this way the positive entries of the table are the gains of player R and the 
losses of player C, and the negative entries are the gains of player C and the losses of player R. 

In this game the players have no control over their moves; each move is determined by chance. However, if each player can decide 
whether he or she wants to play, then each would want to know how much he or she can expect to win or lose over the long term if 
he or she chooses to play. (Later in the section we will discuss this question and also consider a more complicated situation in 
which the players can exercise some control over their moves by varying the sectors of their wheels.) 

Two-Person Zero-Sum Matrix Games 

The game described above is an example of a two-person zero-sum matrix game. The term zero-sum means that in each play of 
the game, the positive gain of one player is equal to the negative gain (loss) of the other player. That is, the sum of the two gains is 
zero. The term matrix game is used to describe a two-person game in which each player has only a finite number of moves, so that 
all possible outcomes of each play, and the corresponding gains of the players, may be displayed in tabular or matrix form, as in 
Table 1. 


In a general game of this type, let player R have m possible moves and let player C have n possible moves. In a play of the game, 
each player makes one of his or her possible moves, and then a payoff is made from player C to player R, depending on the moves. 
Forj = 1, 2, ..., m, and j = 1,2, ..., n, let us set 

= payoff that player C makes to player R if player R 

makes move i and player C makes move j 


This payoff need not be money; it may be any type of commodity to which we can attach a numerical value. As before, if an entry 
ctij is negative, we mean that player C receives a payoff of | from player R. We arrange these mn possible payoffs in the form of 
an mxn matrix 



a n 

3\2 ■ 


a 21 

322 • 

" a 2n 


a m2 ' 

,J a mn 


which we will call the payoff matrix of the game. 


Each player is to make his or her moves on a probabilistic basis. For example, for the game discussed in the introduction, the ratio 
of the area of a sector to the area of the wheel would be the probability that the player makes the move corresponding to that 
sector. Thus, from Figure 11.8.1, we see that player R would make move 2 with probability -j, and player C would make move 2 


with probability A. Jn the general case we make the following definitions: 


Pi = probability that player /?makes move i (i = 1, 2,.. m) 
qj = probability that player C makes move j (j = 1, 2,..., n) 

















It follows from these definitions that 

and 

With the probabilities pi and qj we form two vectors: 

V=[P1 P2 - Pm] and q = 


P\+ P2 I “4 Pm= 1 
q\ I <72 H-1 <7« = 1 


<71 

<72 

<?H 


We call the row vector p the strategy of player R and the column vector q the strategy of player C. For example, from Figure 
11.8.1 we have 

"l 
4 

1 

4 

1 

3 

1 

6 

for the carnival game described earlier. 


p=[I i 1] and ,= 


From the theory of probability, if the probability that player R makes move i is pj, and independently the probability that player C 
makes move j is qp then p^j is the probability that for any one play of the game, player R makes move i and player C makes 
move j. The payoff to player R for such a pair of moves is <3,j. If we multiply each possible payoff by its corresponding probability 
and sum over all possible payoffs, we obtain the expression 

a \lP\<n 4 ai 2 Pm+- + a lf7 piq„ b fl2lP2?l + - + ^mriPm^n 


Equation 1 is a weighted average of the payoffs to player R\ each payoff is weighted according to the probability of its occurrence. 
In the theory of probability, this weighted average is called the expected payoff to player R. It can be shown that if the game is 
played many times, the long-term average payoff per play to player R is given by this expression. We denote this expected payoff 
by E{\>, q) to emphasize the fact that it depends on the strategies of the two players. From the definition of the payoff matrix A and 
the strategies p and q, it can be verified that we may express the expected payoff in matrix notation as 


E(.V, q) = [pi P 2 


Pm] 


an an 

a 2l a 22 
a ni2 


a ln 

a 2n 


01 

#2 


= vM 


( 2 ) 


Because p, q) is the expected payoff to player R, it follows that _ E(\>, q) is the expected payoff to player C. 


Expected Payoff to Player R 


For the carnival game described earlier, we have 


E(v, q) = vM = 



3 

5 

-2 

- f 

"i i r 

-2 

4 

-3 

-4 

.6 3 2. 

6 

-5 

0 

3 


= 4|= 1805. 


















Thus, in the long run, player R can expect to receive an average of about 18 cents from player C in each play of the game. 


So far we have been discussing the situation in which each player has a predetermined strategy. We will now consider the more 
difficult situation in which both players may change their strategies independently. For example, in the game described in the 
introduction, we would allow both players to alter the areas of the sectors of their wheels and thereby control the probabilities of 
their respective moves. This qualitatively changes the nature of the problem and puts us firmly in the field of true game theory. It is 
understood that neither player knows what strategy the other will choose. It is also assumed that each player will make the best 
possible choice of strategy and that the other player knows this. Thus, player R attempts to choose a strategy p such that E(\i, q) is 
as large as possible for the best strategy q that player C can choose; and similarly, player C attempts to choose a strategy q such 
that £(p ; q) is as small as possible for the best strategy p that player R can choose. To see that such choices are actually possible, 
we shall need the following theorem, called the Fundamental Theorem of Two-Person Zero-Sum Games. (The general proof, 
which involves ideas from the theory of linear programming, will be omitted. However, later we shall prove two special cases of 
the theorem.) 

THEOREM 11.8.1 


Fundamental Theorem of Zero-Sum Games 

There exist strategies p * and q * such that 

E(v *, q) >E(V *, q *) <1 *) 

for all strategies p and q. 


The strategies p * and q * in this theorem are the best possible strategies for players R and C, respectively. To see why this is so, 
let v = £(p * , q * )• The left-hand inequality of Equation 3 then reads 

S(p * , q) > v for all strategies q 

This means that if player R chooses the strategy p * , then no matter what strategy q player C chooses, the expected payoff to 
player R will never be below v. Moreover, it is not possible for player R to achieve an expected payoff greater than v. To see why, 
suppose there is some strategy p * * that player R can choose such that 

£(p * * , q) > v for all strategies q 


Then, in particular, 


E {p * * , q) > V 


But this contradicts the right-hand inequality of Equation 3, which requires that v > E(\% * * , q * ). Consequently, the best player 
R can do is prevent his or her expected payoff from falling below the value v. Similarly, the best player C can do is ensure that 
player R's expected payoff does not exceed v, and this can be achieved by using strategy q * . 


On the basis of this discussion, we arrive at the following definitions. 


DEFINITION 


If p * and q * are strategies such that 


for all strategies p and q, then 


£(P * , q) > £(p * , q * ) > £(p, q * ) 


(4) 








(i) ] i * is called an optimal strategy for player R. 

(ii) q * is called an optimal strategy for player C. 

(iii) v = B(\> * , q * ) is called the value of the game. 

The wording in this definition suggests that optimal strategies are not necessarily unique. This is indeed the case, and in Exercise 2 
we ask the reader to show this. However, it can be proved that any two sets of optimal strategies always result in the same value v 
of the game. That is, if p * , q * and p * * , q * * are optimal strategies, then 

£(p*,q*)=£(p* *,q* *) (5) 

The value of a game is thus the expected payoff to player R when both players choose any possible optimal strategies. 

To find optimal strategies, we must find vectors p * and q * that satisfy Equation 4. This is generally done by using linear 
programming techniques. Next, we discuss special cases for which optimal strategies may be found by more elementary 
techniques. 

We now introduce the following definition. 


DEFINITION 


An entry a rs in a payoff matrix A is called a saddle point if 

(i) a r r is the smallest entry in its row, and 

(ii) a r r is the largest entry in its column. 

A game whose payoff matrix has a saddle point is called strictly determined. 


For example, the shaded element in each of the following payoff matrices is a saddle point: 

5 —9l 
-2 10 

6 9 : 

-3 2^ 

If a matrix has a saddle point a rs . it turns out that the following strategies are optimal strategies for the two players: 


p"=[0 0 ... 1 ... 0], q* = 

/ 

i’tJ'j cei trv 


■*— 5th entry 


3 i 

-4 0 


30 -50 -5 

60 90 75 

-10 60 -30 


0 -3 
15 -8 

7 10 

6 11 


That is, an optimal strategy for player R is to always make the rth move, and an optimal strategy for player C is to always make the 
5 th move. Such strategies for which only one move is possible are called pure strategies. Strategies for which more than one move 
















is possible are called mixed strategies. To show that the above pure strategies are optimal, the reader may verify the following 
three equations (see Exercise 6): 


£(i> *, q*) = i> *A* = a rs 


( 6 ) 


7?(p * , q) = p * A(\ > a r r for any strategy q 


(V) 


5(p, q*) = 1>A* <<*rs for any strategy p 


( 8 ) 


Together, these three equations imply that 

£(p *, <l) >5(p *, q * ) >E(\>, q *) 

for all strategies p and q. Because this is exactly Equation 4, it follows that p * and q * are optimal strategies. 

From Equation 6 the value of a strictly determined game is simply the numerical value of a saddle point a r3 . It is possible for a 
payoff matrix to have several saddle points, but then the uniqueness of the value of a game guarantees that the numerical values of 
all saddle points are the same. 


Optimal Strategies to Maximize a Viewing Audience 


Two competing television networks, R and C, are scheduling one-hour programs in the same time period. Network R can schedule 
one of three possible programs, and network C can schedule one of four possible programs. Neither network knows which 
program the other will schedule. Both networks ask the same outside polling agency to give them an estimate of how all possible 
pairings of the programs will divide the viewing audience. The agency gives them each Table 2, whose y)-th entry is the 
percentage of the viewing audience that will watch network R if network R' s program i is paired against network C's program j. 
What program should each network schedule in order to maximize its viewing audience? 

Table 2 

Audience Percentage for Network R 



Network C's Program 

1 

2 

3 

4 


1 

60 

20 

30 

55 

Network R's Program 

2 

50 

75 

45 

60 


3 

70 

45 

35 

30 


Solution 

Subtract 50 from each entry in Table 2 to construct the following matrix: 

'10 - 30 -20 5 

0 25 - 5 10 

20 - 5 - 15-20 



















This is the payoff matrix of the two-person zero-sum game in which each network is considered to start with 50% of the audience, 
and the (j ; y)-th entry of the matrix is the percentage of the viewing audience that network C loses to network R if programs i and j 
are paired against each other. It is easy to see that the entry 

a 2 3= - 5 

is a saddle point of the payoff matrix. Hence, the optimal strategy of network R is to schedule program 2, and the optimal strategy 
of network C is to schedule program 3. This will result in network R 's receiving 45% of the audience and network C's receiving 
55% of the audience. 


2x2 Matrix Games 

Another case in which the optimal strategies can be found by elementary means occurs when each player has only two possible 
moves. In this case, the payoff matrix is a 2 x 2 matrix 

'<*11 ai2~ 


A = 


<*21 <*22 


If the game is strictly determined, at least one of the four entries of A is a saddle point, and the techniques discussed above can then 
be applied to determine optimal strategies for the two players. If the game is not strictly determined, we first compute the expected 
payoff for arbitrary strategies p and q : 


(9) 


"<*11 <*12" 

"7l" 

<*21 <*22 

72 


Because 


5(i>, q) = 1>A = [pi Pi ] 

= <*117+71 +<*127+72+<*217*271 +<*227*272 

7++ 7*2 = 1 and <?i+<?2 = 1 


we may substitute = 1 — p i an( J 72 = 1 — 71 i nt0 9 to obtain 

5(l>, q) = <*117+71 + <*127+0 -<7l) + <*2l0 “J+)7l + <*220 = 7*l)0 -7l) 

If we rearrange the terms in Equation 11, we may write 

5 ( l >, a ) = [ 0*11 4 <*22 “<*12 = <* 2 l ) 7 *l - (<*22 ~ ^ 21 ) ] 71 + (<*12 “ <* 22)^1 + a 22 

By examining the coefficient of the 71 term in 12, we see that if we set 


P 1 =P\ 


<*22 — <*21 


<*11 +- <*22 -<*12 -<*21 


then that coefficient is zero, and 12 reduces to 


n )= _ <* 11<*22 <* 12<*21 _ 

<*11 +<* 22 -<* 12-<*21 


( 10 ) 


(ID 


( 12 ) 


(13) 


(14) 


Equation 14 is independent of 7 ; that is, if player R chooses the strategy determined by 13, player C cannot change the expected 
payoff by varying his or her strategy. 


In a similar manner, it may be verified that if player C chooses the strategy determined by 


* L ?22 — a \2 

7l=7l =-;-^- 

1 <* 11 +<* 22 -<* 12-<*21 


(15) 


then substituting in 12 gives 


q * ) =_ <* 11<*22 — <* 12<*21 


<*11 + <*22 -<*12 -<*21 


(16) 


Equations 14 and 16 show that 


£(l> * , q) = £(l> * , q * ) = 5(1), q * ) 


( 17 ) 












for all strategies p and q. Thus, the strategies determined by 13, 15, and 10 are optimal strategies for players R and C, respectively, 
and so we have the following result. 


THEOREM 11.8.2 


Optimal Strategies for a 2 x 2 Matrix Game 

For a 2x2 game that is not strictly determined, optimal strategies for players R and C are 


and 


p * = 


a 22 — ^21 


fl ll ~ fl 12 


^ 11 +^ 22 -^ 12-^21 ^ 11 +^ 22 -^ 12-^21 


q* 


_ 022 ~ a 12 _ 

a \\ + ^ 22 -^ 12-^21 

_ fl ll ~ a 21 _ 

a\\ + 022 -a\2-<*2\ 


The value of the game is 


v = _ fl ll a 22 — fl 12 fl 21 _ 

an I ^22-^12-^21 


In order to be complete, we must show that the entries in the vectors p * and q * are numbers strictly between 0 and 1. In Exercise 
8 we ask the reader to show that this is the case as long as the game is not strictly determined. 

Equation 17 is interesting in that it implies that either player may force the expected payoff to be the value of the game by choosing 
his or her optimal strategy, regardless of which strategy the other player chooses. This is not true, in general, for games in which 
either player has more than two moves. 


Using Theorem 11.8.2 


The federal government desires to inoculate its citizens against a certain flu virus. The virus has two strains, and the proportions in 
which the two strains occur in the virus population is not known. Two vaccines have been developed. Vaccine 1 is 85% effective 
against strain 1 and 70% effective against strain 2. Vaccine 2 is 60% effective against strain 1 and 90% effective against strain 2. 
What inoculation policy should the government adopt? 


Solution 

We may consider this a two-person game in which player R (the government) desires to make the payoff (the fraction of citizens 
resistant to the virus) as large as possible, and player C (the virus) desires to make the payoff as small as possible. The payoff 
matrix is 


Strain 

I 7 


Vaccine 


.85 .70' 
.60 .90 


This matrix has no saddle points, so Theorem 11.8.2 is applicable. Consequently, 














Pi = n 


?2 


v = 


a 22 -«21 


1 

o 

CO 

1 

o 

0\ 

.30 

2 

All +«22 -<312 -«21 

.85 4- 

.90-.70-.60 ' 

.45 ' 

3 

a 22 — a 12 


.90-.70 

.20 

4 

to 

to 

1 

to 

1 

to 

1 

.85 + 

.90-.70-.60 ' 

.45 ' 

9 

a\\a27-ana21 

(-85) (. 90) — (70) (.60) 

.345 

<*11 +^22-^12-^21 

'-Ti 

CO 

1 .90 -.70 -.60 

.45 


= .7666... 


7 1 

Thus, the optimal strategy for the government is to inoculate of the citizens with vaccine 1 and of the citizens with vaccine 2. 

This will guarantee that about 76.7% of the citizens will be resistant to a virus attack regardless of the distribution of the two 
strains. 


In contrast, a virus distribution of ^ of strain 1 and of strain 2 will result in the same 76.7% of resistant citizens, regardless of 

9 9 

the inoculation strategy adopted by the government (see Exercise 8). 


Exercise Set 11.8 



Click here for Just Ask! 


Suppose that a game has a payoff matrix 

1 . 



6 -4 1 

-7 3 8 

0 6-2 


(a) If players R and C use strategies 



and 


q = 


1 

4 

1 

4 

1 

4 

1 

4 


respectively, what is the expected payoff of the game? 


(b) If player C keeps his strategy fixed as in part (a), what strategy should player R choose to maximize his expected 
payoff? 


(c) If player R keeps her strategy fixed as in part (a), what strategy should player C choose to minimize the expected payoff 
to player R1 


Construct a simple example to show that optimal strategies are not necessarily unique. For example, find a payoff matrix with 
2. several equal saddle points. 
















For the strictly determined games with the following payoff matrices, find optimal strategies for the two players, and find the 

3. values of the games. 



For the 2x2 games with the following payoff matrices, find optimal strategies for the two players, and find the values of the 

4. games. 


(a) r 6 3 
_ — 1 4_ 


(b) r 40 20' 
_ — !0 30_ 



(d) [3 5' 
_5 2_ 



Player R has two playing cards: a black ace and a red four. Player C also has two cards: a black two and a red three. Each player 

5. secretly selects one of his or her cards. If both selected cards are the same color, player C pays player R the sum of the face 
values in dollars. If the cards are different colors, player R pays player C the sum of the face values. What are optimal strategies 
for both players, and what is the value of the game? 




Verify Equations 6 , 7, and 8 . 


6 . 


Verify the statement in the last paragraph of Example 3. 

7. 


Show that the entries of the optimal strategies p * and q * given in Theorem 11.8.2 are numbers strictly between zero and one. 

8 . 


Section 11.8 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Consider a game between two players where each player can make up to n different moves (« > 1). If the z'th move of player 
Tl* R and theyth move of player C are such that; 4 y is even, then C pays R $1. If i f j is odd, then R pays C$1. Assume that 
both players have the same strategy—that is, Vn = [p,] lxH and q H = [p, ] Mxl , where fn + p 2 -\- fn + - + p H = 1- Use a 
computer to show that 

£(l>2, <12) = Ol -P2) 2 
£(P3> <13) = Ol ~P2 + Pi) 2 
£(P4, 14) = Ol -P2 + P3-P4) 2 

U) = Ol ~P2 + P3-P4+ P5) 2 

Using these results as a guide, prove in general that the expected payoff to player R is 

5Cl>».q») = ^EC-l)- /+1 

which shows that in the long run, player R will not lose in this game. 

Consider a game between two players where each player can make up to n different moves (« > 1). If both players make th 
same move, then player C pays player R $(«_!). However, if both players make different moves, then player R pays pla; 
C $1. Assume that both players have the same strategy—that is, j and q H = [p,] j, where 

P! + p 2 + p 2 |_h p M = 1- Use a computer to show that 



T2. 



£(P2> 12) = ^(pl -p\) 2 + ^(Pl -P2) 2 I -^(P2-P\) 2 I ^(P2-P2) 2 
£(P3> 13) = Tj-Ol - Pi) 2 4 ^-(pi -P2) 2 4 -^(pi -P3) 2 

+ -^(P2-‘P1) 2 I 2"(p2 — P2) 2 I " 2 "(P2 — P3) 2 
+ 2 "(P3 — Pi) 2 + "2 C^3 — P2) 2 I "2 (P3 — P3) 2 
^(P4, M4) = "2 (Pi — Pi) 2 + 2"(P1 — P2) 2 I 2 "(pl — P3) 2 I' "2 (Pi — P4) 2 

+ -^(P2-Pl ) 2 I ~2 (P 2 — P2 ) 2 4 (p2 — P3 ) 2 + 2 (P 2 — P 4) 2 

+ ^(P3-Pl) 2 + ^CP3-P2) 2 I ^(P3-P3) 2 +^(P3-P4) 2 

4 ^-(p4-pi) 2 I ^(P4-P2) 2 + ^CP4-P3) 2 + ^(P4-P4) 2 

Using these results as a guide, prove in general that the expected payoff to player R is 

S(v„,<i n ) = ±±±(p i ~p J ) 2 >0 
2 i=lj =1 

which shows that in the long run, player R will not lose in this game. 
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11.9 

LEONTIEF ECONOMIC 
MODELS 


In this section we discuss two linear models for economic systems. Some 
results about nonnegative matrices are applied to determine equilibrium price 
structures and outputs necessary to satisfy demand. 


Linear Systems 
Matrices 


Economic Systems 

Matrix theory has been very successful in describing the interrelations among prices, outputs, and demands in economic systems. 
In this section we discuss some simple models based on the ideas of Nobel laureate Wassily Leontief. We examine two different 
but related models: the closed or input-output model, and the open or production model. In each, we are given certain economic 
parameters that describe the interrelations between the “industries” in the economy under consideration. Using matrix theory, we 
then evaluate certain other parameters, such as prices or output levels, in order to satisfy a desired economic objective. We begin 
with the closed model. 

Leontief Closed (Input-Output) Model 

First we present a simple example; then we proceed to the general theory of the model. 


An Input-Output Model 


Three homeowners—a carpenter, an electrician, and a plumber—agree to make repairs in their three homes. They agree to work a 
total of 10 days each according to the following schedule: 



Work Performed by 



Carpenter 

Electrician Plumber 

Days of Work in Home of Carpenter 

2 

1 

6 

Days of Work in Home of Electrician 

4 

5 

1 

Days of Work in Home of Plumber 

4 

4 

3 


For tax purposes, they must report and pay each other a reasonable daily wage, even for the work each does on his or her own 
home. Their normal daily wages are about $100, but they agree to adjust their respective daily wages so that each homeowner will 
come out even—that is, so that the total amount paid out by each is the same as the total amount each receives. We can set 

p\ = daily wage of carpenter 
P 2 = daily wage of electrician 
P 2 = daily wage of plumber 

To satisfy the “equilibrium” condition that each homeowner comes out even, we require that 











total expenditures = total income 

for each of the homeowners for the 10-day period. For example, the carpenter pays a total of 2^1 H p 2 4 &P 3 f° r the repairs in his 
own home and receives a total income of \Qp\ for the repairs that he performs on all three homes. Equating these two expressions 
then gives the first of the following three equations: 

1 4 P2 + &P3 = 10j?i 
4pi 4 5p 2 + P3 = \0p2 
4pi H 4p2 + 3p 3 = 10 p2 

The remaining two equations are the equilibrium equations for the electrician and the plumber. Dividing these equations by 10 and 
rewriting them in matrix form yields 


( 1 ) 


'.2 .1 

.6' 

>f 


>f 

.4 .5 

.1 

P2 

= 

P2 

1 - 

4 ^ 

4 ^ 

.3 

P3 


P3 


Equation 1 can be rewritten as a homogeneous system by subtracting the left side from the right side to obtain 


.8 

-.1 

-.6' 

>f 


'O' 

-.4 

.5 

-.1 

P2 

= 

0 

-.4 

-.4 

.7 

P3 


0 


The solution of this homogeneous system is found to be (verify) 


>r 


"31" 

P2 

= s 

32 

P3 


36 


where 5 is an arbitrary constant. This constant is a scale factor, which the homeowners may choose for their convenience. For 
example, they may set s = 3 so that the corresponding daily wages—$93, $96, and $108—are about $100. 

4 

This example illustrates the salient features of the Leontief input-output model of a closed economy. In the basic Equation 1, each 
column sum of the coefficient matrix is 1, corresponding to the fact that each of the homeowners' “output” of labor is completely 
distributed among these same homeowners in the proportions given by the entries in the column. Our problem is to determine 
suitable “prices” for these outputs so as to put the system in equilibrium—that is, so that each homeowner's total expenditures 
equal his or her total income. 

In the general model we have an economic system consisting of a finite number of “industries,” which we number as industries 1, 
2, ..., k. Over some fixed period of time, each industry produces an “output” of some good or service that is completely utilized in 
a predetermined manner by the k industries. An important problem is to find suitable “prices” to be charged for these k outputs so 
that for each industry, total expenditures equal total income. Such a price structure represents an equilibrium position for the 
economy. 

For the fixed time period in question, let us set 

p i = price charged by the /'th industry for its total output 

Sjj = fraction of the total output of the jth industry purchased by the /'th industry 
for i, j = \,2,k. By definition, we have 

0) Pi>0, / = 1, 2,..., k 


(ii) su> 0, i,j, =1,2 ,k 


(iii) e\j 4- e 2 j + - -I- e k .j = 1, j= 1,2,.... k 


With these quantities, we form the price vector 


















and the exchange matrix or input-output matrix 


Pi 




Ph 



"*11 

*12 - 

*lfc 

E = 

*21 

*22 - 

.... to 


*Jcl 

*fc2 - 

*Jcfc 


Condition (iii) expresses the fact that all the column sums of the exchange matrix are 1. 

As in the example, in order that the expenditures of each industry be equal to its income, the following matrix equation must be 
satisfied [see 1]: 


E V = p 


( 2 ) 


or 


(/-5)p = 0 (3) 

Equation 3 is a homogeneous linear system for the price vector p. It will have a nontrivial solution if and only if the determinant of 
its coefficient matrix / _ £ is zero. In Exercise 7 we ask the reader to show that this is the case for any exchange matrix E. Thus, 3 
always has nontrivial solutions for the price vector/;. 

Actually, for our economic model to make sense, we need more than just the fact that 3 has nontrivial solutions for p. We also need 
the prices pj of the k outputs to be nonnegative numbers. We express this condition as p > 0. (In general, if A is any vector or 
matrix, the notation A > 0 means that every entry of A is nonnegative, and the notation A > Q means that every entry of A is 
positive. Similarly, A > B means A — B > 0, and A> B means A — B> Q0-) To show that 3 has a nontrivial solution for which p > 0 
is a bit more difficult than showing merely that some nontrivial solution exists. But it is true, and we state this fact without proof in 
the following theorem. 


THEOREM 11.9.1 


IfE is an exchange matrix, then Ep = p always has a nontrivial solution p whose entries are nonnegative. 


Let us consider a few simple examples of this theorem. 


Using Theorem 11.9.1 


Let 


Then (/ — E)\> = 0 is 


1 

0 




2 

'pi' 


'o' 

1 

0 

P2 


0 

2 





which has the general solution 

















where s is an arbitrary constant. We then have nontrivial solutions p > 0 for any s > 0- 


Using Theorem 11.9.1 


Let 


Then (/ — 5)p = 0 has the general solution 



0 

1 


P = s 



0 

1 


where s and t are independent arbitrary constants. Nontrivial solutions p > 0 then result from any s > 0 and t > 0, not both zero. 


Example 2 indicates that in some situations one of the prices must be zero in order to satisfy the equilibrium condition. Example 3 
indicates that there may be several linearly independent price structures available. Neither of these situations describes a truly 
interdependent economic structure. The following theorem gives sufficient conditions for both cases to be excluded. 


THEOREM 11.9.2 


Let E be an exchange matrix such that for some positive integer m all the entries of E m are positive. Then there is exactly one 
linearly independent solution of (I — £')p = 0, and it may be chosen so that all its entries are positive. 


We will not give a proof of this theorem. The reader who has read Section 11.6 on Markov chains may observe that this theorem is 
essentially the same as Theorem 11.6.4. What we are calling exchange matrices in this section were called stochastic or Markov 
matrices in Section 11.6. 


Using Theorem 11.9.2 


The exchange matrix in Example 1 was 


E = 


.2 .1 
.4 .5 
.4 .4 


.6 

.1 

.3 


Because E > 0> the condition E m > 0 i n Theorem 11.9.2 is satisfied for rn = \- Consequently, we are guaranteed that there is 
exactly one linearly independent solution of (/ — £)p = 0, and it can be chosen so that p > 0- In that example, we found that 


P = 


31 

32 
36 


is such a solution. 


♦ 


Leontief Open (Production) Model 



















In contrast with the closed model, in which the outputs of k industries are distributed only among themselves, the open model 
attempts to satisfy an outside demand for the outputs. Portions of these outputs may still be distributed among the industries 
themselves, to keep them operating, but there is to be some excess, some net production, with which to satisfy the outside demand. 
In the closed model the outputs of the industries are fixed, and our objective is to determine prices for these outputs so that the 
equilibrium condition, that expenditures equal incomes, is satisfied. In the open model it is the prices that are fixed, and our 
objective is to determine levels of the outputs of the industries needed to satisfy the outside demand. We will measure the levels of 
the outputs in terms of their economic values using the fixed prices. To be precise, over some fixed period of time, let 

x 3 - = monetary value of the total output of the ith industry 

= monetary value of the output of the ith industry needed to satisfy the outside demand 
Cij = monetary value of the output of the ith industry needed by the y'th industry to produce one unit of monetary value of 
its own output 


With these quantities, we define the production vector 


the demand vector 


*1 

*2 


d 1 



dk 


and the consumption matrix 


By their nature, we have that 



cil 

c 12 

- Clfc 

c= 

C 21 

c 22 

Cl 

... to 


Cfcl 

c k2 

c kk 

x > 0, 

tl>0, 

and 


From the definition of and xj, it can be seen that the quantity 

C]l*l + c i2 x 2 H-b CiftJfc 

is the value of the output of the ith industry needed by all k industries to produce a total output specified by the production vector 
x. Because this quantity is simply the ith entry of the column vector Cx> we can say further that the ith entry of the column vector 

x-Cx 

is the value of the excess output of the ith industry available to satisfy the outside demand. The value of the outside demand for the 
output of the ith industry is the ith entry of the demand vector d. Consequently, we are led to the following equation 

x — Cx= (l 


or 


(/ — C)x = (l 


(4) 


for the demand to be exactly met, without any surpluses or shortages. Thus, given C and d, our objective is to find a production 
vector x > 0 that satisfies Equation 4. 


Production Vector for a Town 


A town has three main industries: a coal-mining operation, an electric power-generating plant, and a local railroad. To mine $1 of 
coal, the mining operation must purchase $.25 of electricity to run its equipment and $.25 of transportation for its shipping needs. 
To produce $1 of electricity, the generating plant requires $.65 of coal for fuel, $.05 of its own electricity to run auxiliary 










equipment, and $.05 of transportation. To provide $1 of transportation, the railroad requires $.55 of coal for fuel and $.10 of 
electricity for its auxiliary equipment. In a certain week the coal-mining operation receives orders for $50,000 of coal from outside 
the town, and the generating plant receives orders for $25,000 of electricity from outside. There is no outside demand for the local 
railroad. How much must each of the three industries produce in that week to exactly satisfy their own demand and the outside 
demand? 


Solution 


For the one-week period let 

x i = value of total output of coal-mining operation 
X 2 = value of total output of power-generating plant 
*3 = value of total output of local railroad 


From the information supplied, the consumption matrix of the system is 


The linear system (/ — C)x = d is then 


C = 


0 

.25 

.25 


.65 

.05 

.05 


.55 

.10 

0 


1.00 

-.65 

-.55' 

'*f 


'50,000' 

-.25 

.95 

-.10 

x 2 

= 

25,000 

-.25 

-.05 

1.00 

*3 


0 


The coefficient matrix on the left is invertible, and the solution is given by 


x = (/-C) _1 d = 


'756 542 470' 

'50,000' 


'102,087' 

- 220 690 190 

25,000 

= 

56,163 

200 170 630 

0 


28,330 


Thus, the total output of the coal-mining operation should be $102,087, the total output of the power-generating plant should be 
$56,163, and the total output of the railroad should be $28,330. 


Let us reconsider Equation 4: 

If the square matrix / _ C is invertible, we can write 


(/ — C)x= d 


x= (I — C) -1 d 


(5) 


In addition, if the matrix (/ — C) 1 has only nonnegative entries, then we are guaranteed that for any d > 0, Equation 5 has a 

unique nonnegative solution for x. This is a particularly desirable situation, as it means that any outside demand can be met. The 
terminology used to describe this case is given in the following definition. 


DEFINITION 


A consumption matrix C is said to be productive if (/ — C) 1 exists and 

(/-C) _1 >0 


We will now consider some simple criteria that guarantee that a consumption matrix is productive. The first is given in the 
following theorem. 






















THEOREM 11.9.3 


Productive Consumption Matrix 

A consumption matrix C is productive if and only if there is some production vector x > 0 such that K > Cx- 


(The proof is outlined in Exercise 9.) The condition x > Cx means that there is some production schedule possible such that each 
industry produces more than it consumes. 

Theorem 11.9.3 has two interesting corollaries. Suppose that all the row sums of C are less than 1. If 

'f 

x= j 
1 

then Cx is a column vector whose entries are these row sums. Therefore, K > Cx, and the condition of Theorem 11.9.3 is satisfied. 
Thus, we arrive at the following corollary: 

COROLLARY 11.9.4 


A consumption matrix is productive if each of its row sums is less than 1. 


As we ask the reader to show in Exercise 8, this corollary leads to the following: 

COROLLARY 11.9.5 


A consumption matrix is productive if each of its column sums is less than 1. 


Recalling the definition of the entries of the consumption matrix C, we see that the y'th column sum of C is the total value of the 
outputs of all k industries needed to produce one unit of value of output of the y'th industry. The y'th industry is thus said to be 
profitable if that y'th column sum is less than 1. In other words. Corollary 11.9.5 says that a consumption matrix is productive if all 
k industries in the economic system are profitable. 


Using Corollary 11.9.5 


The consumption matrix in Example 5 was 



.65 .55 
.05 .10 
.05 0 


All three column sums in this matrix are less than 1 and so all three industries are profitable. Consequently, by Corollary 11.9.5, 
the consumption matrix C is productive. This can also be seen in the calculations in Example 5, as (/ — C) -1 is nonnegative. 



Click here for J 


W # 1 













For the following exchange matrices, find nonnegative price vectors that satisfy the equilibrium condition 3. 

1 . 


(a) 


! I 
2 3 

1 1 

2 3 


(b) 


1 

2 

1 

3 

i_ 

6 


0 

0 


1 

2 

1 

2 


1 0 


35 

.50 

.30 

25 

.20 

.30 

.40 

.30 

.40 


Using Theorem 11.9.3 and its corollaries, show that each of the following consumption matrices is productive. 

2 . 


(a) 


.8 .1 
.3 .6 


70 

.30 

.25 

20 

.40 

.25 

.05 

.15 

.25 


(c) 


.7 .3 .2 
.1 .4 .3 
.2 .4 .1 


Using Theorem 11.9.2, show that there is only one linearly independent price vector for the closed economic system with 
3. exchange matrix 


E = 


0 

1 

0 


.2 

.2 

.6 


.5 

.5 

0 


Three neighbors have backyard vegetable gardens. Neighbor A grows tomatoes, neighbor B grows corn, and neighbor C grows 
lettuce. They agree to divide their crops among themselves as follows: A gets -i of the tomatoes, of the corn, and of the 

lettuce. B gets -j of the tomatoes, -j of the com, and of the lettuce. C gets ^ of the tomatoes, -j of the com, and ^ of the 

lettuce. What prices should the neighbors assign to their respective crops if the equilibrium condition of a closed economy is to 
be satisfied, and if the lowest-priced crop is to have a price of $ 100 ? 

Three engineers—a (CE), an (EE), and a (ME)—each have a consulting firm. The consulting they do is of a multidisciplinary 
5. nature, so they buy a portion of each others' services. For each $1 of consulting the CE does, she buys $.10 of the EE's services 
and $.30 of the ME's services. For each $1 of consulting the EE does, she buys $.20 of the CE's services and $.40 of the ME's 
services. And for each $1 of consulting the ME does, she buys $.30 of the CE's services and $.40 of the EE's services. In a 
certain week the CE receives outside consulting orders of $500, the EE receives outside consulting orders of $700, and the ME 
receives outside consulting orders of $600. What dollar amount of consulting does each engineer perform in that week? 






6 . 


(a) Suppose that the demand d, for the output of the ith industry increases by one unit. Explain why the /'th column of the 
matrix (I — C) _1 is the increase that must be made to the production vector x to satisfy this additional demand. 

(b) Referring to Example 5, use the result in part (a) to determine the increase in the value of the output of the coal-mining 
operation needed to satisfy a demand of one additional unit in the value of the output of the power-generating plant. 

Using the fact that the column sums of an exchange matrix E are all 1, show that the column sums of / _ ff are zero. From this, 

7. show that I — E has zero determinant, and so (1 — E)\> = 0 has nontrivial solutions for p. 

Show that Corollary 11.9.5 follows from Corollary 11.9.4. 

8 . 

—1 T 

Hint Use the fact that (A T ) = (A -1 ) f° r an Y invertible matrix A. 

9. (For Readers Who Have Studied Calculus) Prove Theorem 11.9.3 as follows: 

(a) Prove the “only if ” part of the theorem; that is, show that if C is a productive consumption matrix, then there is a vectc 
x > 0 such that x > Cx- 

(b) Prove the “if ” part of the theorem as follows: 

Step 1. Show that if there is a vector x * > 0 such that Cx * < x * > then x * > 0- 

Step 2. Show that there is a number A such that 0 < A < 1 an d Cx * < Ax * • 

Step 3. Show that C H x * < A”x * . 

Step 4. Show that C M —* 0 as n —> l30 • 

Step 5. By multiplying out, show that 

(I -C)(I A C + C 2 4-h C M-1 ) =I-C ” 

for n = 1, 2 ,.., 

Step 6. By letting n —, , X i in Step 5, show that the matrix infinite sum 

S = 1 I C 4 C 2 E- 

exists and that ( I-C)S=I- 

Step 7. Show that S > 0 and that S = (1 — C ) -1 . 



Step 8. Show that C is a productive consumption matrix. 


Section 11.9 


Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Tl. 


Consider a sequence of exchange matrices {£ 2 , £ 3 , E 4 , E$,E n ), where 


£2 = 




[0 1 il 

[0 II 


2 3 

2 

1 1 

5 3 = 

1 0 j 

2 


0 1 1 



2 3 


E 4 = 




"0 i i i i’ 

2 3 4 5 

1 0 - - - 

3 4 5 

0 1 0 1 1 

2 4 5 

"0 1 1 1" 

2 3 4 

1 0 3 4 

0 1 0 1 

. ^5 = 

2 4 

° H 


0 0 1 0 1 

3 5 

0 0 0 | 1 



4 5 


and so on. Use a computer to show that gi q g3 q g4 q gj q and make the conjecture that although . o m is 
true, gk .. q is not true for k = 1, 2, 3, — 1. Next, use a computer to determine the vectors \t n such that ,, = p H (for 

n = 2, 3, 4, 5, 6 ), and then see if you can discover a pattern that would allow you to compute p H+ i easily from p H . Test your 
discovery by first constructing pg from 


P7 = 


and then checking to see whether Egpg = pg- 


2520 

3360 

1890 

672 

175 

36 

7 


T2. 


Consider an open production model having n industries with n > ]. In order to produce $1 of its own output, the jth industr 
must spend $ (1 /«) for the output of the ith industry (for all i * j, but the jth industry (for all j = 1, 2, 3,..., n) spends 
nothing for its own output. Construct the consumption matrix c , show that it is productive, and determine an expression f 
(/ _ C n ) -1 . In determining an expression for (/ _ c „) _1 , use a computer to study the cases when « = 2, 3, 4, and 5; the 
make a conjecture and prove your conjecture to be true. 


Hint If p = r 1 ] w (that is, the n'x.n matrix with every entry equal to 1), first show that 

** ^ J r t Af J 












and then express your value of in terms of n, / H , and p „■ 
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11.10 

FOREST MANAGEMENT 


In this section we discuss a matrix model for the management of a forest where 
trees are grouped into classes according to height. The optimal sustainable 
yield of aperiodic harvest is calculated when the trees of different height 
classes can have different economic values. 


Matrix Operations 


Optimal Sustainable Yield 

Our objective is to introduce a simplified model for the sustainable harvesting of a forest whose trees are classified by height. The 
height of a tree is assumed to determine its economic value when it is cut down and sold. Initially, there is a distribution of trees of 
various heights. The forest is then allowed to grow for a certain period of time, after which some of the trees of various heights are 
harvested. The trees left unharvested are to be of the same height configuration as the original forest, so that the harvest is 
sustainable. As we will see, there are many such sustainable harvesting procedures. We want to find one for which the total 
economic value of all the trees removed is as large as possible. This determines the optimal sustainable yield of the forest and is 
the largest yield that can be attained continually without depleting the forest. 

The Model 

Suppose that a harvester has a forest of Douglas fir trees that are to be sold as Christmas trees year after year. Every December the 
harvester cuts down some of the trees to be sold. For each tree cut down, a seedling is planted in its place. In this way the total 
number of trees in the forest is always the same. (In this simplified model, we will not take into account trees that die between 
harvests. We assume that every seedling planted survives and grows until it is harvested.) 

In the marketplace, trees of different heights have different economic values. Suppose that there are n different price classes 
corresponding to certain height intervals, as shown in Table 1 and Figure 11.10.1. The first class consists of seedlings with heights 
in the interval [0, k\), and these seedlings are of no economic value. The nth class consists of trees with heights greater than or 
equal to k n -\- 


Table 1 


Class 

Value (dollars) 

Height Interval 

1 (seedings) 

None 

[0, h i) 

2 

P2 


3 

P3 

[^2’ ^ 3 ) 

: : = 

n — 1 

Pyi —1 

2’ 

n 

Pyi 

00 ) 












Let Xj (i = 1 , 2 ,be the number of trees within the zth class that remain after each harvest. We form a column vector with the 
numbers and call it the nonharvest vector. 

~x\~ 

x 2 

x = 

x n 


For a sustainable harvesting policy, the forest is to be returned after each harvest to the fixed configuration given by the nonharvest 
vector x. Part of our problem is to find those nonharvest vectors x for which sustainable harvesting is possible. 


Because the total number of trees in the forest is fixed, we can set 

x\+X2 I- =s 


( 1 ) 


where s is predetermined by the amount of land available and the amount of space each tree requires. Referring to Figure 11.10.2, 
we have the following situation. The forest configuration is given by the vector x after each harvest. Between harvests the trees 
grow and produce a new forest configuration before each harvest. A certain number of trees are removed from each class at the 
harvest. Finally, a seedling is planted in place of each tree removed, to return the forest again to the configuration x. 



Forest before growth horcst after harvest 

(nonharvesL vector x] (rronftarvGSt vecior x) 


Figure 11.10.2 


Consider first the growth of the forest between harvests. During this period a tree in the zth class may grow and move up to a 
higher height class. Or its growth may be retarded for some reason, and it will remain in the same class. We consequently define 
the following growth parameters gj for i = 1 , 2 ,..., n — 1 : 

gj = the fraction of trees in the zth class that grow into the (j |_ ])-st class during a growth period 


For simplicity we assume that a tree can move at most one height class upward in one growth period. With this assumption, we 


















have 


1 — g, = the fraction of trees in the ith class that remain in the z'th class during a growth period 


With these n — \ growth parameters, we form the following nxn growth matrix-. 


a= 


i -gi 
si 
o 

0 

0 


0 

1 -S2 


S 2 1 - S3 


0 

0 

0 

1 — En —1 0 
gn-l 1 


(2) 


Because the entries of the vector x are the numbers of trees in the n classes before the growth period, the reader can verify that the 
entries of the vector 

(1 -gi)*i 
£ 1*1 4 0 - g 2)*2 
£2*2 f 0 “g3)*3 


Gx — 

I 

£m—2*h—2 I 0 — £m— l)*w—1 
£h—I^h— 1 i 

are the numbers of trees in the n classes after the growth period. 

Suppose that during the harvest we remove y,(; = 1,2trees from the ith class. We will call the column vector 

>1 

72 

7h 


(3) 


the harvest vector. Thus, a total of 


71 -1 72 f —t 7 h 


trees are removed at each harvest. This is also the total number of trees added to the first class (the new seedlings) after each 
harvest. If we define the following nx.n replacement matrix 


R = 


11-1 


0 0 


0 


0 0-0 


(4) 


then the column vector 


i?v = 


71 I 72 


7m 


(5) 


specifies the configuration of trees planted after each harvest. 


At this point we are ready to write the following equation, which characterizes a sustainable harvesting policy: 


configuration 


11 ‘ 


configuration 

at end of 

— [harvest] f 

new seedling 

— 

at beginning of 

growth period 

replacement 


growth period 


or, mathematically, 

Gx — y - 1 - Rv — x 



This equation can be rewritten as 


(I-R)y=(G-l)x 


( 6 ) 


or, more comprehensively, 


'o 

-1 

-1 ■* 

-1 

-1" 

’ y i 


-g\ 

0 

0 ... 

0 

o' 

" *1 

0 

1 

0 

0 

0 

y2 


g\ 

-g.2 

0 ... 

0 

0 

*2 

0 

0 

1 ... 

0 

0 

73 

= 

0 

g2 

-g 3 ... 

0 

0 

*3 

0 

0 

0 - 

1 

0 

yn -1 


0 

0 

0 - 

~ gn—1 

0 

—1 

0 

0 

0 - 

0 

1 

y» 


0 

0 

0 - 

gn—1 

0 



We will refer to Equation 6 as the sustainable harvesting condition. Any vectors x and y with nonnegative entries, and such that 

*1 4 x 2 H-h x n = s, which satisfy this matrix equation determine a sustainable harvesting policy for the forest. Note that if 

yj > 0, then the harvester is removing seedlings of no economic value and replacing them with new seedlings. Because there is no 
point in doing this, we assume that 


y\ = 0 


With this assumption, it can be verified that 6 is the matrix form of the following set of equations: 

72+73 4 - \ yn=g\*l 

72=gl*l — £2*2 
73 =£2*2-£3*3 


7w—1 — gn— 2*m— 2 gn— 1*m—1 
y n = gn— 1*h—1 

Note that the first equation in 8 is the sum of the remaining n — \ equations. 


Because we must have y i > 0 for i = 2, 3,..., n. Equations 8 require that 


(7) 


( 8 ) 


£ 1*1 >£ 2 * 2 >->£w- 1 *w -1 >0 

Conversely, if x is a column vector with nonnegative entries that satisfy Equation 9, then 7 and 8 define a column vector y with 
nonnegative entries. Furthermore, x and y then satisfy the sustainable harvesting condition 6 . In other words, a necessary and 
sufficient condition for a nonnegative column vector x to determine a forest configuration that is capable of sustainable harvesting 
is that its entries satisfy 9. 

Optimal Sustainable Yield 

Because we remove y i trees from the z'th class (j = 2,3,..., n) and each tree in the zth class has an economic value of p i the total 
yield of the harvest, Yld, is given by 


Yld = piy 2 4 P3y3 4-b Pny n 


( 10 ) 


Using 8 , we may substitute for the y,'s in 10 to obtain 

Yld=p2g\x\ f (P3 — P2)g2 x 2 4— 4- (Pn ~ Pn-l)g n -\x n -\ 


(ID 


Combining 11,1, and 9, we can now state the problem of maximizing the yield of the forest over all possible sustainable 
harvesting policies as follows: 


Find nonnegative numbers x j, x 2 , ■■■, x n that maximize 


Yld=p 2 gix\ 4- (P3~P2)g2*2-\ -b (;?„)g„_ix„_i 










subject to 


*1 + *2 H-1- * H = s 


and 

£ 1*1 >£ 2 * 2 > ■■■>£»-! * w -1 >0 

As fonnulated above, this problem belongs to the field of linear programming. However, we shall illustrate the following result 
below, without linear programming theory, by actually exhibiting a sustainable harvesting policy. 


THEOREM 11.10.1 


Optimal Sustainable Yield 

The optimal sustainable yield is achieved by harvesting all the trees from one particular height class and none of the trees from 
any other height class. 


Let us first set 

Yldfr = yield obtained by harvesting all of the Arth class and none of the other classes 

The largest value of yidk for k = 2,2,..., n will then be the optimal sustainable yield, and the corresponding value of k will be the 
class that should be completely harvested to attain the optimal sustainable yield. Because no class but the kth is harvested, we have 

y2=y2 = -=yk-\ =^^+i =-=^ M = o , X2 ) 

In addition, because all of the Mi class is harvested, no trees are left unharvested in the Mi class, and no trees are ever present in 
the height classes above the Mi class. Thus, 


*fc — *fc+l— L “ — *»— 0 (13) 

Substituting 12 and 13 into the sustainable harvesting condition 8 gives 

yk = g\x\ 

0=gl*l — £2*2 
0 = g2*2 -£3*3 

: (14) 

0 = gfc-2*fc-2 ~ gfc-l*fc-l 
yk = Ek— i*fc —i 

Equations 14 can also be written as 


yh=E\x\ =g2*2 : 


Ek— l*ft—1 


from which it follows that 


If we substitute Equations 13 and 16 into 

[which is Equation 1], we can solve for jq and obtain 


*2 =£1*1 / E2 
*3=gl*l tg3 

*fc-1 =gl*l tgk-l 
*1 + *2 H-h * H =s 


H — + — H-h 


E2 E3 


Ek -1 


(15) 


(16) 


*1 = 


(17) 






For the yield Yld k , we combine 10, 12, 15, and 17 to obtain 

Wd k = pzy 2 + P2E2 + - + Pn?n 

= pfcyk 


= PkZ 1*1 


EM 


S\ §2 


Sk -1 


( 18 ) 


Equation 18 determines Yld k in terms of the known growth and economic parameters for any k = 2, 3,..., m. Thus, the optimal 
sustainable yield is found as follows. 


THEOREM 11.10.2 


Finding the Optimal Sustainable Yield 

The optimal sustainable yield is the largest value of 


EM. 


f+ f+ ■ 

gl Z2 


■ + ■ 


Zk -1 


for k=2,3,n. The corresponding value of k is the number of the class that is completely harvested. 


In Exercise 4 we ask the reader to show that the nonharvest vector x for the optimal sustainable yield is 


1 fzi 

1 f Z2 


x = 


-^4- -L + - 

gl Z2 




Zk -1 


1 fgk-l 
0 
0 

0 


(19) 


Theorem 11.10.2 implies that it is not necessarily the highest-priced class of trees that should be totally cropped. The growth 
parameters g 3 - must also be taken into account to determine the optimal sustainable yield. 


Using Theorem 11.10.2 


For a Scots pine forest in Scotland with a growth period of six years, the following growth matrix was found (see M. B. Usher, “A 
Matrix Approach to the Management of Renewable Resources, with Special Reference to Selection Forests,” Journal of Applied 
Ecology, vol. 3, 1966, pp. 355-367): 


.72 0 0 0 0 0 

.28 .69 0 0 0 0 

0 .31 .75 0 0 0 

0 0 .25 .77 0 0 

0 0 0 .23 .63 0 

0 0 0 0 .37 1.00 


Suppose that the prices of trees in the five tallest height classes are 

P 7 = $50, ?i = $100, ^4= $150, 


p*, = $ 200 , ^^ = $250 


Which class should be completely harvested to obtain the optimal sustainable yield, and what is that yield? 















Solution 


From the matrix G we have that 


£i = -28, £3 = .31, £1 = .25, £4 = .23, £^ = .37 

Equation 18 then gives 

YId 2 = 50sf ( 28 -1 ) = 14.0s 

YId 3 = 100sl (,28 _1 + .31 -1 ) = 14.7s 

Yld A = 150s/(.28 _1 f ,31 _1 4- ,25 _1 ) = 13.9s 

m 5 = 200s/ (.28 —1 f ,31 -1 + ,25 -1 4- ,23 _1 ) = 13.2s 

Yld§ = 250s/ (.28 —1 f.31 _1 + ,25 _1 4- ,23 _1 + .37 -1 ) = 14.0s 

We see that Yld 3 is the largest of these five quantities, so from Theorem 11.10.2 the third class should be completely harvested 
every six years to maximize the sustainable yield. The corresponding optimal sustainable yield is $ 14.7s, where s is the total 
number of trees in the forest. 


♦ 


Exercise Set 11.10 



Click here for Just Ask! 


A certain forest is divided into three height classes and has a growth matrix between harvests given by 

0 0 


If the price of trees in the second class is $30 and the price of trees in the third class is $50, which class should be completely 
harvested to attain the optimal sustainable yield? What is the optimal yield if there are 1000 trees in the forest? 

In Example 1, to what level must the price of trees in the fifth class rise so that the fifth class is the one to harvest completely in 
2. order to attain the optimal sustainable yield? 

In Example 1, what must the ratio of the prices p 2 : p 3 : p^: p*;. p fi be in order that the yields Yld^ k = 2,3, 4, 5, 6, all be the 
3* same? (In this case, any sustainable harvesting policy will produce the same optimal sustainable yield.) 

Derive Equation 19 for the nonharvest vector x corresponding to the optimal sustainable harvesting policy described in 

4. Theorem 11.10.2. 

For the optimal sustainable harvesting policy described in Theorem 11.10.2, how many trees are removed from the forest 

5. during each harvest? 

If all the growth parameters g\, g 2 , g H -i in the growth matrix G are equal, what should the ratio of the prices p 2 : p 3 : ■■■'■ p n 

6. be in order that any sustainable harvesting policy be an optimal sustainable harvesting policy? (See Exercise 3.) 


1 

2 



0 






Section 11.10 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


A particular forest has growth parameters given by 

Ei = \ 

for j = 1,2,3 . n — 1, where n (the total number of height classes) can be chosen as large as needed. Suppose that the value 

of a tree in the Mi height interval is given by 

p k = a(k- l) f 

where a is a constant (in dollars) and p is a parameter satisfying 1 < p < 2. 


(a) Show that the yield Yld k is given by 


Yld k = 


2a(k_ — l) p-1 g 

k 


(b) For 


p=1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9 

use a computer to determine the class number that should be completely harvested, and 
determine the optimal sustainable yield in each case. Make sure that you allow k to take on 
only integer values in your calculations. 

(c) Repeat the calculations in part (b) using 

p= 1.91, 1.92, 1.93, 1.94, 1.95, 1.96, 1.97, 1.98, 1.99 

(d) Show that if p = 2, then the optimal sustainable yield can never be larger than 2 as- 


(e) Compare the values of k determined in parts (b) and (c) to 1 / (2 — p), and use some calculus to explain why 


k ^ 


1 

2 -P 


A particular forest has growth parameters given by 


for i = 1, 2, 3,..., >2—1, where n (the total number of height classes) can be chosen as large as needed. Suppose that the val 
of a tree in the Mi height interval is given by 


p k = a(k-\) p 




where a is a constant (in dollars) and p is a parameter satisfying 1 < p. 


(a) Show that the yield Yld^ is given by 


Yld k = 


a(£ — 1) 1 g 
2 k -2 


(b) For 


p= 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 

use a computer to determine the class number that should be completely harvested in order 
to obtain an optimal yield, and determine the optimal sustainable yield in each case. Make 
sure that you allow k to take on only integer values in your calculations. 

(c) Compare the values of k determined in part (b) to l |_ p j ln(2) and use some calculus to explain why 
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11.11 

COMPUTER GRAPHICS 


In this section we assume that a view of a three-dimensional object is displayed 
on a video screen and show how matrix algebra can be used to obtain new 
views of the object by rotation, translation, and scaling. 


Matrix Algebra 
Analytic Geometry 


Visualization of a Three-Dimensional Object 

Suppose that we want to visualize a three-dimensional object by displaying various views of it on a video screen. The object we 
have in mind to display is to be determined by a finite number of straight line segments. As an example, consider the truncated 
right pyramid with hexagonal base illustrated in Figure 11.11.1. We first introduce an xyz-coordinate system in which to embed 
the object. As in Figure 11.11.1, we orient the coordinate system so that its origin is at the center of the video screen and the xy 
-plane coincides with the plane of the screen. Consequently, an observer will see only the projection of the view of the 
three-dimensional object onto the two-dimensional xy-plane. 



Figure 11.11.1 

In the xyz-coordinate system, the endpoints p^, ...,/> of the straight line segments that determine the view of the object will 

have certain coordinates—say, 

(xuy\,z\), (x7,y?.,Z7),..., (x„,y„,z„) 

These coordinates, together with a specification of which pairs are to be connected by straight line segments, are to be stored in the 
memory of the video display system. For example, assume that the 12 vertices of the truncated pyramid in Figure 11.11.1 have the 
following coordinates (the screen is 4 units wide by 3 units high): 


Pv 

(1.000, - 

-.800,.000), 

Py 

(.500, 

-.800, - .866), 

Py- 

(- 

.500, 

-.800, - .866), 

Pa- 

(- 1.000, - .800, .000), 

Py 

C- 

.500, 

-.800, .866), 

Py 

U-| 

o 

o 

-.800, .866), 

Pr 

II 

o 

^r 

co 

.400, .000), 

Py 

(.315,. 

125, - .546), 

Py 

(- 

.210,. 

.650, - .364), 

^10 

: ( — .360, .300, .000), 

^ii 

: (- 

-.210, 

.650,.364), 

Pl2 

: (.315, 

.125, .546), 


These 12 vertices are connected pairwise by 18 straight line segments as follows, where P i ^ p, denotes that point p i is connected 
to point p,: 







P2 ++Pa> Pa^ P5, P^^Pf,, P^^P\, 

P-j+*Pg, Pgr-yPg, Pg^P\a, -?10 ■*-» ^ 11 , ^ 11 ^^ 12 . P\2^Pl> 

P\r->P-], P 2 r--P%, P3M-P9, Pa^P\q, PjT-tfn, Pf>^Pn> 

In View 1 these 18 straight line segments are shown as they would appear on the video screen. It should be noticed that only the x- 
and y-coordinates of the vertices are needed by the video display system to draw the view, because only the projection of the object 
onto the ^y-plane is displayed. However, we must keep track of the ^-coordinates to carry out certain transformations discussed 
later. 



View 1 


We now show how to form new views of the object by scaling, translating, or rotating the initial view. We first construct a 3 x n 
matrix P, referred to as the coordinate matrix of the view, whose columns are the coordinates of the n points of a view: 



1- L 

* 

to 






p= 

VI V 2 

Vm 






z\ z 2 - 






the coordinate matrix P corresponding to View 1 is the 3 

X 12 

matrix 




1.000 .500 -.500 - 1.000 -.500 

.500 

.840 

.315 

-.210 

-.360 

-.210 .315 

O 

O 

CO 

1 

0 

0 

CO 

II' 

0 

0 

CO 

r 

0 

0 

CO 

1 

0 

0 

CO 

1 

-.800 - 

.400 

.125 

.650 

.800 

.650 .125 

.000 -.866 -.866 .000 .866 

.866 

.000 

-.546 

-.364 

.000 

.364 .546 


We will show below how to transform the coordinate matrix P of a view to a new coordinate matrix p' corresponding to a new 
view of the object. The straight line segments connecting the various points move with the points as they are transformed. In this 
way, each view is uniquely determined by its coordinate matrix once we have specified which pairs of points in the original view 
are to be connected by straight lines. 


Scaling 


The first type of transformation we consider consists of scaling a view along the *, y, and z directions by factors of q, 3, and 7, 
respectively. By this we mean that if a point p i has coordinates (x lr y lr z{) in the original view, it is to move to a new point p{ with 
coordinates (axj, 3y u r \z } ) in the new view. This has the effect of transforming a unit cube in the original view to a rectangular 
parallelepiped of dimensions a x ,3 x 7 (Figure 11.11.2). Mathematically, this may be accomplished with matrix multiplication as 
follows. Define a 3 x 3 diagonal matrix 


Q 0 



£ 

0 


0 

0 


7 

















' 

I 




Figure 11.11.2 

Then, if a point p i in the original view is represented by the column vector 


yi 

z i 


then the transformed point p* is represented by the column vector 


x i 


a 

0 

o' 


y\ 

= 

0 

,3 

0 

yi 



0 

0 

7 

Zi 


Using the coordinate matrix P, which contains the coordinates of all n points of the original view as its columns, we can transform 
these n points simultaneously to produce the coordinate matrix P ! of the scaled view, as follows: 


SP = 


a 0 0 
0 8 0 
0 0 7 

0*1 ax 2 

&y\ Pyi 

7Z1 7Z2 


1 - 

'^1 

X2 ■ 

,J x n 


71 

72 ' 


1 

z\ 

z 2 • 

" z n 


ax„ 

Py» 


=p ! 


The new coordinate matrix can then be entered into the video display system to produce the new view of the object. As an 
example. View 2 is View 1 scaled by setting a = ] g, 8 = 0.5, and 7 = 3.0. Note that the scaling 7 = 3.0 along the z-axis is not 
visible in View 2, since we see only the projection of the object onto the xy-plane. 



-1 































View 2 

View 1 scaled by a = 1 . 8 , /? = 0.5, 7 = 3.0. 


Translation 

We next consider the transformation of translating or displacing an object to a new position on the screen. Referring to Figure 
11.11.3, suppose we desire to change an existing view so that each point p i with coordinates (x lr y lr z{) moves to a new point pj 
with coordinates (*,. -|- XQi Vl 4 . yQi z , -|- Z[l ), The vector 

'* 0 ' 

70 

ZQ 



Figure 11.11.3 

is called the translation vector of the transformation. By defining a 3 x n matrix Fas 


*0 

*0 ■ 

- x 0 

70 

70 ‘ 

" 70 

ZQ 

ZQ ■ 

'• z 0 


we can translate all n points of the view determined by the coordinate matrix P by matrix addition via the equation 

P' = P I- 7 

The coordinate matrix p ! then specifies the new coordinates of the n points. For example, if we wish to translate View 1 according 
to the translation vector 

' 1 . 2 ' 

0.4 

1.7 

the result is View 3. Note, again, that the translation Z q = 1.7 along the z-axis does not show up explicitly in View 3. 



View 3 

View 1 translated by = 1 . 2 , — 0.4, z 0 = 1-7- 


In Exercise 7, a technique of performing translations by matrix multiplication rather than by matrix addition is explained. 

Rotation 


A more complicated type of transformation is a rotation of a view about one of the three coordinate axes. We begin with a rotation 



















about the z-axis (the axis perpendicular to the screen) through an angle #. Given a point p i in the original view with coordinates 
(x u y u Zj), we wish to compute the new coordinates ( x ! y{ z {^ of the rotated point . Referring to Figure 11.11.4 and using a 
little trigonometry, the reader should be able to derive the following: 

xj- = p cos {6 + #) = p cos 6 cos 0 — p sin0 sin # = x,cos 9 — ^sin# 
y'l — p sin (6 4 - 9) — p cos ^ sin 9 + p sin 6 cos 9 — Xj-sin^ 4- y ;• cos 9 


= Zi 



These equations can be written in matrix form as 


v~ 




x i 


cos# —sin# 0 



y'i 

= 

sin# cos# 0 


yi 

j 


0 0 1 


Zi 

z i 






If we let R denote the 3 x 3 matrix in this equation, all n points can be rotated by the matrix product 

p ! =rp 

to yield the coordinate matrix P ! of the rotated view. 


Rotations about the x- and y-axes can be accomplished analogously, and the resulting rotation matrices are given with Views View 
4, View 5, and View 6 . These three new views of the truncated pyramid correspond to rotations of View 1 about the x-, y-, and 
z-axes, respectively, each through an angle of 90°. 



View 4 

View 1 rotated 90° about the x-axis. 



View 5 


View 1 rotated 90° about the y-axis. 








































View 6 

View 1 rotated 90° about the z-axis. 


Rotations about three coordinate axes may be combined to give oblique views of an object. For example, View 7 is View 1 rotated 
first about the v-axis through 30°, then about the y-axis through -70°, and finally about the "-axis through -21°. Mathematically, 
these three successive rotations can be embodied in the single transformation equation p ! = RP RP, where R is the product of three 
individual rotation matrices: 


R\ = 


R'i = 


l o o 

0 cos(30°) — sin(30°) 

0 sin(30°) cos(30°) 

cos( — 70°) 0 sin( —70°) 

0 1 0 


in the order 



— sin( 

-70°) 

0 cos(—70 

°) 


cos( — 

27°) - 

- sm( - 27°) 

o' 

*3 = 

sin( — 

27°) 

cos( — 27°) 

0 


0 


0 

1 



.305 

-.025 - 

.952 

R = R^R^R 1 = 

-.155 

.985 - 

.076 



.940 

.171 

.296 



View 7 

Oblique view of truncated pyramid. 


As a final illustration, in View 8 we have two separate views of the truncated pyramid, which constitute a stereoscopic pair. They 
were produced by first rotating View 7 about the y-axis through an angle of -3° and translating it to the right, then rotating the 
same View 7 about the y-axis through an angle of +3° and translating it to the left. The translation distances were chosen so that 
the stereoscopic views are about 2-^- inches apart—the approximate distance between a pair of eyes. 
































View 8 

Stereoscopic figure of truncated pyramid. The three-dimensionality of the diagram can be seen by holding the book 
about one foot away and focusing on a distant object. Then by shifting gaze to View 8 without refocusing, you can make 
the two views of the stereoscopic pair merge together and produce the desired effect. 


Exercise Set 11.11 



Click here for Just Ask! 


1 . 


View 9 is a view of a square with vertices (0, 0, 0), (1, 0, 0), (1, 1, 0), and (0, 1, 0). 
(a) What is the coordinate matrix of View 9? 


(b) 


What is the coordinate matrix of View 9 after it is scaled by a factor 
a sketch of the scaled view. 


llin the x-direction ad -i in the y-direction? Dra' 


-2 -I 0 I 



View 9 

Square with vertices (0, 0, 0), (1, 0, 0), (1, 1, 0), and (0, 1, 0) (Exercises 1 and 2). 


(c) What is the coordinate matrix of View 9 after it is translated by the following vector? 

" — 2 " 

- 1 
3 


Draw a sketch of the translated view. 





















(d) What is the coordinate matrix of View 9 after it is rotated through an angle of -30° about the z-axis? Draw a sketch of 
the rotated view. 


(a) If the coordinate matrix of View 9 is multiplied by the matrix 

I »' 

0 1 0 
0 0 1 

the result is the coordinate matrix of View 10. Such a transformation is called a shear in the 
x-direction with factor with respect to the y-coordinate. Show that under such a 

transformation, a point with coordinates (x u y u zi) has new coordinates + ly u y uZ .y 


o i 




/ 

/ 






View 10 


View 9 sheared along the x-axis by -i with respect to the y-coordinate (Exercise 2). 


(b) What are the coordinates of the four vertices of the shear square in View 10? 


(c) The matrix 


1 0 0 
.6 1 0 
0 0 1 

determines a shear in the y-direction with factor .6 with respect to the x-coordinate (an 
example appears in View 11). Sketch a view of the square in View 9 after such a shearing 
transformation, and find the new coordinates of its four vertices. 



View 11 


View 1 sheared along the y-axis by .6 with respect to the x-coordinate (Exercise 2). 






















3 . 


(a) The reflection about the xz-plane is defined as the transformation that takes a point (x u y u z,) to the point (x u — 7 ,-, zf) 
(e.g.. View 12). If P and P ! are the coordinate matrices of a view and its reflection about the plane, respectively, find 
a matrix M such that P ! = MP- 



View 12 

View 1 reflected about the ^plane (Exercise 3). 


(b) Analogous to part (a), define the reflection about the yz-plane and construct the corresponding transformation matrix. 
Draw a sketch of View 1 reflected about the yz-plane. 


(c) Analogous to part (a), define the reflection about the xy-plane and construct the corresponding transformation matrix. 
Draw a sketch of View 1 reflected about the xy-plane. 


4. 


(a) View 13 is View 1 subject to the following five transformations: 

Scale by a factor of in the x-direction, 2 in the y-direction, and -i in the z-direction. 

1*2 3 


Translate unit in the x-direction. 

2 . 2 


Rotate 20° about the x-axis. 


3. 

Rotate -45° about the y-axis. 

4. 

Rotate 90° about the z-axis. 

5. 

Construct the five matrices My M 2 ’ M 3 ’ M$> and associated with these five transformations. 

- 2-1012 

1 

0 



-I 




















(b) 


View 13 

View 1 scaled, translated, and rotated (Exercise 4). 

If P is the coordinate matrix of View 1 and p' is the coordinate matrix of View 13, express p* in terms of My My My 
MMy and P. 


5. 

(a) View 14 is View 1 subject to the following seven transformations: 

Scale by a factor of .3 in the x-direction and by a factor of .5 in the y-direction. 

1 . 


Rotate 45° about the x-axis. 

2 . 


Translate 1 unit in the x-direction. 

3. 


Rotate 35° about the y-axis. 

4. 


Rotate -45° about the z-axis. 

5. 


Translate 1 unit in the z-direction. 

6 . 


Scale by a factor of 2 in the x-direction. 

7. 


Construct the matrices My My<---' M-j associated with these seven transformations. 



View 14 

View 1 scaled, translated, and rotated (Exercise 5). 


(b) If P is the coordinate matrix of View 1 and P ! is the coordinate matrix of View 14, express P' in terms of MyMy---’ 
M 7 , and P. 


Suppose that a view with coordinate matrix P is to be rotated through an angle f/ about an axis through the origin and specifiec 
6. by two angles a and 3 (see Figure Ex-6). If P ! is the coordinate matrix of the rotated view, find rotation matrices P>y R 

and p^ such that 


P ! = R 5 R^R 2 R\P 













Figure Ex-6 

Hint The desired rotation can be accomplished in the following five steps: 
Rotate through an angle of 3 about the y-axis. 

Rotate through an angle of a about the z-axis. 

3 Rotate through an angle of 0 about the v-axis. 

Rotate through an angle of — a about the z-axis. 

5 Rotate through an angle of — 3 about the y-axis. 


This exercise illustrates a technique for translating a point with coordinates (x u y u z,) to a point with coordinates 
(jf | | j’q, z, -|- zf|) by matrix multiplication rather than matrix addition. 


(a) Let the point (x u y u Zl ) be associated with the column vector 


v, = 


*! 
yi 
Zi 
l 

and let the point (>, + x 0 , jv, -i yo,Zj-\ zq ) be associated with the column vector 

’*!+ X 0 ‘ 


v' = 


yi+y o 

Zi + ZQ 

l 


Find a 4x4 matrix M such that v ' = M\ v 


(b) Find the specific 4x4 matrix of the above form that will effect the translation of the point (4, -2, 3) to the point (-1,7, 

0 ). 


For the three rotation matrices given with Views View 4, View 5, and View 6, show that 

R~ { =R T 


(A matrix with this property is called an orthogonal matrix. See Section 6.5.) 









Section 11.11 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Let (a, b, c) be a unit vector normal to the plane ax + by -4 cz = Cf and let r = (x, y, z ) be a vector. It can be shown that the 
TL mirror image of the vector r through the above plane has coordinates i-,,, = y mr where 


with 


X?n 


~x " 

y m 

= M 




z 


M = I- 2im r 


'1 

0 

o' 


~a~ 

0 

1 

0 

-2 

b 

0 

0 

1 


c 


(a) Show that m 2 = / and give a physical reason why this must be so. 
Hint Use the fact that (a, b, c) is a unit vector to show that n ^ n = 

(b) Use a computer to show that det(il/) = — 1- 


(c) The eigenvectors of M satisfy the equation 




~x " 


~x " 


= M 


= A 


z m 


z 


z 


and therefore correspond to those vectors whose direction is not affected by a reflection 
through the plane. Use a computer to determine the eigenvectors and eigenvalues of M, and 
then give a physical argument to support your answer. 

A vector y = (x, y, z) is rotated by an angle 0 about an axis having unit vector (a, b, c), thereby forming the rotated vector 
v R = (x R , y R , z R )- It can be shown that 




~x " 

yp. 

=m 


ZR 


z 


with 




















R(G) =cos(&) 


[a b c ] 


'1 

0 

o' 


~a~ 

0 

1 

0 

+ (l-cos( 0 )) 

h 

0 

0 

1 


c 


I sin(#) 


0 — c b 

c 0 —a 

— b a 0 


(a) Use a computer to show that Bs,U)R{S) = R(9 \ , and then give a physical reason why this must be so. Depending 

on the sophistication of the computer you are using, you may have to experiment using different values of a, b, and 

c= f\ -a 2 -b 2 

(b) Show also that R -1 (9) =R{—9 ) and give a physical reason why this must be so. 


(c) Use a computer to show that det(.fi(0)) = + 1 . 
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11.12 

EQUILIBRIUM 

TEMPERATURE 

DISTRIBUTIONS 


In this section we shall see that the equilibrium temperature distribution within a 
trapezoidal plate can be found when the temperatures around the edges of the 
plate are specified. The problem is reduced to solving a system of linear 
equations. Also, an interactive technique for solving the problem and a “random 
walk” approach to the problem are described. 


Linear Systems 
Matrices 

Intuitive Understanding of Limits 


Boundary Data 


Suppose that the two faces of the thin trapezoidal plate shown in Figure 11.12.1a are insulated from heat. Suppose that we are also 
given the temperature along the four edges of the plate. For example, let the temperature be constant on each edge with values of 
0°, 0°, 1°, and 2°, as in the figure. After a period of time, the temperature inside the plate will stabilize. Our objective in this 
section is to determine this equilibrium temperature distribution at the points inside the plate. As we will see, the interior 
equilibrium temperature is completely determined by the boundary data —that is, the temperature along the edges of the plate. 






1.00 


n.<m 


Figure 11.12.1 


The equilibrium temperature distribution can be visualized by the use of curves that connect points of equal temperature. Such 
curves are called isotherms of the temperature distribution. In Figure 11.12.1 b we have sketched a few isotherms, using 
information we derive later in the chapter. 

Although all our calculations will be for the trapezoidal plate illustrated, our techniques generalize easily to a plate of any practical 
shape. They also generalize to the problem of finding the temperature within a three-dimensional body. In fact, our “plate” could 
be the cross section of some solid object if the flow of heat perpendicular to the cross section is negligible. For example, Figure 
11.12.1 could represent the cross section of a long dam. The dam is exposed to three different temperatures: the temperature of the 
ground at its base, the temperature of the water on one side, and the temperature of the air on the other side. A knowledge of the 
temperature distribution inside the dam is necessary to determine the thermal stresses to which it is subjected. 


Next, we shall consider a certain thermodynamic principle that characterizes the temperature distribution we are seeking. 

The Mean-Value Property 



















There are many different ways to obtain a mathematical model for our problem. The approach we use is based on the following 
property of equilibrium temperature distributions. 


THEOREM 11.12.1 


The Mean-Value Property 

Let a plate be in thermal equilibrium and let P be a point inside the plate. Then if C is any circle with center at P that is 
completely contained in the plate, the temperature at P is the average value of the temperature on the circle ( Figure 11.12.2). 



This property is a consequence of certain basic laws of molecular motion, and we will not attempt to derive it. Basically, this 
property states that in equilibrium, thermal energy tends to distribute itself as evenly as possible consistent with the boundary 
conditions. It can be shown that the mean-value property uniquely determines the equilibrium temperature distribution of a plate. 

Unfortunately, determining the equilibrium temperature distribution from the meanvalue property is not an easy matter. However, 
if we restrict ourselves to finding the temperature only at a finite set of points within the plate, the problem can be reduced to 
solving a linear system. We pursue this idea next. 

Discrete Formulation of the Problem 


We can overlay our trapezoidal plate with a succession of finer and finer square nets or meshes (Figure 11.12.3). In (a) we have a 
rather coarse net; in ( b ) we have a net with half the spacing as in (a)\ and in (c) we have a net with the spacing again reduced by 
half. The points of intersection of the net lines are called mesh points. We classify them as boundary mesh points if they fall on the 
boundary of the plate or as interior mesh points if they lie in the interior of the plate. For the three net spacings we have chosen, 
there are 1, 9, and 49 interior mesh points, respectively. 



(ff) (interior mesh point 

Figure 11.12.3 



( b ) 9 i nicriur mesh poiills 



111(11111 
( C ) 49 into nor wicssh points 


0 

0 

0 

0 













































In the discrete formulation of our problem, we try to find the temperature only at the interior mesh points of some particular net. 
For a rather fine net, as in (c), this will provide an excellent picture of the temperature distribution throughout the entire plate. 

At the boundary mesh points, the temperature is given by the boundary data. (In Figure 11.12.3 we have labeled all the boundary 
mesh points with their corresponding temperatures.) At the interior mesh points, we shall apply the following discrete version of 
the mean-value property. 


THEOREM 11.12.2 


Discrete Mean-Value Property 

At each interior mesh point, the temperature is approximately the average of the temperatures at the four neighboring mesh 
points. 


This discrete version is a reasonable approximation to the true mean-value property. But because it is only an approximation, it 
will provide only an approximation to the true temperatures at the interior mesh points. However, the approximations will get 
better as the mesh spacing decreases. In fact, as the mesh spacing approaches zero, the approximations approach the exact 
temperature distribution, a fact proved in advanced courses in numerical analysis. We will illustrate this convergence by computing 
the approximate temperatures at the mesh points for the three mesh spacings given in Figure 11.12.3. 

Case (a) of Figure 11.12.3 is simple, for there is only one interior mesh point. If we let ^ be the temperature at this mesh point, the 
discrete mean-value property immediately gives 

; 0 = I(2 + 1 + 0 + 0) = 0.75 


In case ( b ) we can label the temperatures at the nine interior mesh points t\, ■■■■> t$, as in Figure 11.12.3 b. (The particular 

ordering is not important.) By applying the discrete mean-value property successively to each of these nine mesh points, we obtain 
the following nine equations: 



+ 2 

1 0 I 

0) 

*2 = ^1 

+ *3 

+ £4 

1 2) 

*3 = ^2 

+ h 

+ 0 I 

0) 


+ h 

All 

1 2) 

*5 = 4^3 

+ £4 

+ £t 

1 <s) 


+ £9 

+ 0 1 

0) 

*7 = ^(*4 

+ £s 

+ H 

2) 

*8 =4 

+ £7 

+ £9 

1 1) 

£9 = 

+ *8 

+ n 

0) 


This is a system of nine linear equations in nine unknowns. We can rewrite it in matrix form as 


t = Mt + h 


( 2 ) 


where 





h 

ti 


1 3 
t A 


t = 


h 


£6 


*■1 





To solve Equation 2, we write it as 


The solution for t is thus 


M = 


o 4 


0 


0 
0 

0 0 


4 0 0 4 


0 0 0 0 0 0 0 
0 0 0 0 0 
0 0 0 0 


1 1 
4 4 


4oo4o4oo 


1 1 
4 4 


1 

4 


4 4 o 4 o 4 o 


1 

4 


1 


1 

4 


0 0 0 0 
0 0 0 0 0 


1 

4 


oooo4ooo4 


0004000 

4 


1 

4 


1 

4 


4 0 


(I-M) t = b 


t= (7-ik0 -1 h 



(3) 


as long as the matrix (/ _ is invertible. This is indeed the case, and the solution for t as calculated by 3 is 


t = 


0.7846 

1.1383 

0.4719 

1.2967 

0.7491 

0.3265 

1.2995 

0.9014 

0.5570 


(4) 


Figure 11.12.4 is a diagram of the plate with the nine interior mesh points labeled with their temperatures as given by this solution. 



Figure 11.12.4 































For case (c) of Figure 11.12.3, we repeat this same procedure. We label the temperatures at the 49 interior mesh points as •• 
*49 

in some manner. For example, we may begin at the top of the plate and proceed from left to right along each row of mesh points. 
Applying the discrete mean-value property to each mesh point gives a system of 49 linear equations in 49 unknowns: 

*1 — 4 (*2 + 24-0 + 0) 

*2 = ^(*l+*3 + *4+2) 


(5) 


*43 = ^(*41 + *47 + *49+1) 

*49 = ^"(*42 + *48 I 0-I-1) 

In matrix form, Equations 5 are 

t = Mt + b 

where t and b are column vectors with 49 entries, and M is a 49 x 49 matrix. As in 3, the solution for t is 


t= (/- 


( 6 ) 


In Figure 11.12.5we display the temperatures at the 49 mesh points found by Equation 6. The nine unshaded temperatures in this 
figure fall on the mesh points of Figure 11.12.4. In Table 1 we compare the temperatures at these nine common mesh points for the 
three different mesh spacings used. 




































Figure 11.12.5 


Table 1 



Temperatures at Common Mesh Points 


Case (a) 

Case ( b ) 

Case (c) 


— 

0.7846 

0.8048 

^2 

— 

1.1383 

1.1533 

*3 

— 

0.4719 

0.4778 

*4 

— 

1.2967 

1.3078 

h 

0.7500 

0.7491 

0.7513 


— 

0.3265 

0.3157 

£7 

— 

1.2995 

1.3042 


— 

0.9014 

0.9032 

tg 

— 

0.5570 

0.5554 


Knowing that the temperatures of the discrete problem approach the exact temperatures as the mesh spacing decreases, we may 
surmise that the nine temperatures obtained in case (c) are closer to the exact values than those in case (b). 

A Numerical Technique 

To obtain the 49 temperatures in case (c) of Figure 11.12.3, it was necessary to solve a linear system with 49 unknowns. A finer 
net might involve a linear system with hundreds or even thousands of unknowns. Exact algorithms for the solutions of such large 
systems are impractical, and for this reason we nowdiscuss a numerical technique for the practical solution of these systems. 

To describe this technique, we look again at Equation (2): 


t = Mt 4- b ( 7 ) 

The vector t we are seeking appears on both sides of this equation. We consider a way of generating better and better 
approximations to the vector solution t. For the initial approximation we may take j-C 1 - 1 ) = o if no better choice is available. If we 
substitute into the right side of 7 and label the resulting left side as t-0), we have 


= Mt® + b ( 

Usually fC 1 ) is a better approximation to the solution than is t®. If we substitute fO) into the right side of 7, we generate another 
approximation, which we label fl): 


4 b 


( 9 ) 


Continuing in this way, we generate a sequence of approximations as follows: 







( 10 ) 


= M# 1 + b 
t® = I b 

= I b 

t (») = i, 


One would hope that this sequence of approximations j-GO, ^0), ^(2), ... converges to the exact solution of 7. We do not have the 
space here to go into the theoretical considerations necessary to show this. Suffice it to say that for the particular problem we are 
considering, the sequence converges to the exact solution for any mesh size and for any initial approximation t®. 


This technique of generating successive approximations to the solution of 7 is a variation of a technique called Jacobi iteration ; 
the approximations themselves are called iterates. As a numerical example, let us apply Jacobi iteration to the calculation of the 
nine mesh point temperatures of case ( b ). Setting = 0> we have, from Equation 2, 




,5C 

500 

.51 

500 

,0( 

500 

.51 

500 

,0( 

500 

,0( 

500 




Some additional iterates are 



All iterates beginning with the thirtieth are equal to fCO) to four decimal places. Consequently, is the exact solution to four 















































decimal places. This agrees with our previous result given in Equation 4. 

The Jacobi iteration scheme applied to the linear system 5 with 49 unknowns produces iterates that begin repeating to four decimal 
places after 119 iterations. Thus, would provide the 49 temperatures of case (c) correct to four decimal places. 


A Monte Carlo Technique 

In this section we describe a so-called Monte Carlo technique for computing the temperature at a single interior mesh point of the 
discrete problem without having to compute the temperatures at the remaining interior mesh points. First we define a discrete 
random walk along the net. By this we mean a directed path along the net lines (Figure 11.12.6) that joins a succession of mesh 
points such that the direction of departure from each mesh point is chosen at random. Each of the four possible directions of 
departure from each mesh point along the path is to be equally probable. 



Figure 11.12.6 


By the use of random walks, we can compute the temperature at a specified interior mesh point on the basis of the following 
property. 


THEOREM 11.12.3 


Random Walk Property 

Let •••> W„ be a succession of random walks, all of which begin at a specified interior mesh point. Let f*, f*, be 

the temperatures at the boundary mesh points first encountered along each of these random walks. Then the average value 

(f* | i* _|_| i* 'j / n of these boundary temperatures approaches the temperature at the specified interior mesh point as the 

number of random walks n increases without bound. 


This property is a consequence of the discrete mean-value property that the mesh point temperatures satisfy. The proof of the 
random walk property involves elementary concepts from probability theory, and we will not give it here. 

In Table 2 we display the results of a large number of computer-generated random walks for the evaluation of the temperature ^ of 
the nine-point mesh of case ( b ) in Figure 11.12.6. The first column lists the number n of the random walk. The second column lists 
the temperature of the boundary point first encountered along the corresponding random walk. The last column contains the 

cumulative average of the boundary temperatures encountered along the n random walks. Thus, after 1000 random walks we have 
the approximation ^ ^ 7550 - This compares with the exact value ^ = .7491 that we had previously evaluated. As can be seen, the 
convergence to the exact value is not too rapid. 


Table 2 














n 

(^i H-h t n ) / W 

i 

1 

1.0000 

2 

2 

1.5000 

3 

1 

1.3333 

4 

0 

1.0000 

5 

2 

1.2000 

6 

0 

1.0000 

7 

2 

1.1429 

8 

0 

1.0000 

9 

2 

1.1111 

10 

0 

1.0000 

20 

1 

0.9500 

30 

0 

0.8000 

40 

0 

0.8250 

50 

2 

0.8400 

100 

0 

0.8300 

150 

1 

0.8000 

200 

0 

0.8050 

250 

1 

0.8240 

500 

1 

0.7860 

1000 

0 

0.7550 


Exercise Set 11.12 



Click here for Just Ask! 


1 . 


A plate in the form of a circular disk has boundary temperatures of 0° on the left of its circumference and 1° on the right half c 
its circumference. A net with four interior mesh points is overlaid on the disk (see Figure Ex-1). 









(a) Using the discrete mean-value property, write the 4 x 4 linear system t = Mt H b that determines the approximate 
temperatures at the four interior mesh points. 

(b) Solve the linear system in part (a). 

(c) Use the Jacobi iteration scheme with = 0 to generate the iterates (-0), ft), ft), and ft) for the linear system in 
part (a). What is the “error vector” — t, where t is the solution found in part (b)? 

(d) By certain advanced methods, it can be determined that the exact temperatures to four decimal places at the four mesh 
points are t j = 2871 and ^ = 1.4 = .7129- What are the percentage errors in the values found in part (b)? 



Use Theorem 11.12.1 to find the exact equilibrium temperature at the center of the disk in Exercise 1. 

2 . 


3 . 


Calculate the first two iterates ft) and t 1 - 2 ) for case (b) of Figure 11.12.3 with nine interior mesh points [Equation (2)] when the 
initial iterate is chosen as 


t®=[l 1 1 1 1 1 1 1 11 r 


The random walk illustrated in Figure Ex-4a can be described by six arrows 

«- l —♦ —► t —♦ 

that specify the directions of departure from the successive mesh points along the path. Figure Ex-4 b is an array of 100 
computer-generated, randomly oriented arrows arranged in a 10 x 10 array. Use these arrows to determine random walks to 
approximate the temperature as in Table 2. Proceed as follows: 


Take the last two digits of your telephone number. Use the last digit to specify a row and the other to specify a column. 

2 , Go to the arrow in the array with that row and column number. 

3 Using this arrow as a starting point, move through the array of arrows as you would read a book (left to right and top to 
bottom). Beginning at the point labeled ^ in Figure Ex-4a and using this sequence of arrows to specify a sequence of 
directions, move from mesh point to mesh point until you reach a boundary mesh point. This completes your first rando 
walk. Record the temperature at the boundary mesh point. (If you reach the end of the arrow array, continue with the 
arrow in the upper left corner.) 








Return to the interior mesh point labeled ^ and begin where you left off in the arrow array; generate your next random 
walk. Repeat this process until you have completed 10 random walks and have recorded 10 boundary temperatures. 

5 . Calculate the average of the 10 boundary temperatures recorded. (The exact value is ^ = .7491-) 
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Section 11.12 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Tl. 


Suppose that we have the square region described by 

n= {(x, y) |0 < x < 1 , 0 <y < 1 } 


and suppose that the equilibrium temperature distribution u (x, y) along the boundary is given by u (x, 0) = T%, u(x, 1) = 1 
u(0, 7 ) = Ti, and w(l, 7 ) = Tp : Suppose next that this region is partitioned into an (« 4 . ]) x {n + 1) mesh using 

Xi = — and v,= — 

1 n n 

for j = 0, 1,2, ...,/; and j = 0, 1, 2,. .., n. If the temperatures of the interior mesh points are labeled by 

u it j = u{x i ,y i ) = u{iin,jin) 


then show that 


u i,] — ^ ( u i—lj T u !+lj T u i,j —1 “b u !j+l) 

for i = 1, 2, 3, — 1 and j = 1, 2, 3,n — 1. To handle the boundary points, define 

= u„j = T r , u u0 = T b , and u u „ = T t 

for j = 1, 2, 3 ,n — 1 and j = 1, 2, 3,— 1. Next let 













T2. 


be the (n 4 1 ) x (« 4 1 ) matrix with the nxn identity matrix in the upper right-hand corner, a one in the lower left-hand 
corner, and zeros everywhere else. For example, 


?2 = 


1 

'0 

1 

o' 

> ^3 = 

0 

0 

1 

- 

1 

0 

0 


^4 = 


0 10 0 
0 0 10 
0 0 0 1 
10 0 0 


1 

’0 

1 

0 

0 

0" 


0 

0 

1 

0 

0 

. ^5 = 

0 

0 

0 

1 

0 


0 

0 

0 

0 

1 

- 

1 

0 

0 

0 

0 


and so on. By defining the ( n -\ 1 ) x {n 4 1) matrix 

-^ m 41 = ^ m 41 4 F 


0 /„ 
1 0 


4 


0 /, 
1 0 


T 


show that if U n +\ is the («+l)x(« + l) matrix with entries then the set of equations 

u Uj = 4 (“t-lj w i'4lJ +■ u i,j -1 + u !,f4l) 
for i= 1, 2, 3,— 1 and j = 1, 2, 3,1 can be written as the matrix equation 

&n+l = £^h41 "F ^h41^m41) 

where we consider only those elements of U n+ \ with i = 1, 2, 3, ,.„n — 1 and j = 1, 2, 3, 1. 


The results of the preceding exercise and the discussion in the text suggest the following algorithm for solving for the 
equilibrium temperature in the square region 

n = {(*, 7) 10 < * < 1, o<j< 1} 


given the boundary conditions 

u(x,0) = T b , u{x,\) = Tj, h(0 ,y)=T L , u(\,y) = T R 


Choose a value for n, and then choose an initial guess, say 



0 

Tl - 

■ Tl 

0 

Tb 

0 ■■ 

■■ 0 

Tt 

Tb 

0 

■ 0 

Tt 

0 

Tr ■ 

■ Tr 

0 


For each value of k = 0> 1, 2, 3, ..., compute 7 ■/*+h using 

H + l 

X/4+ 1 ) _\ (M r/ k ) . r/K) M 

^H4l — 4 C^h41 ^m41 + ^m41-^h41) 

where is as defined in Exercise Tl. Then adjust by replacing all edge entries by t 

initial edge entries in jy® . 

Note The edge entries of a matrix are the entries in the first and last columns and first and last rows. 


3 . 


Continue this process until ttC^+O j/' k ) 

U n+1 


is approximately the zero matrix. This suggests that 


















T3. 


£/ h+1 = Hm zr„ 


(k) 


+1 


Use a computer and this algorithm to solve for u(x, y) given that 

u{x, 0) = 0, u{x, 1) = 0, w( 0 , 7 )= 0 , u{\,y) = 2 

Choose « = 6 and compute up to jr/ 30 ). The exact solution can be expressed as 

/ v \ = 8 v- sinh[(2wi- l)7rx]sm[(2ws - l)gj/] 

’ y) {2m- l)sinh[(2m- l)z] 

Use a computer to compute u {i / 6 , j / 6 ) for i, j = 0, 1, 2, 3, 4, 5, 6 , and then compare your results to the values of 
u{i/6,jf6) in ca¬ 


using the exact solution u (x, 7 ) for the temperature distribution described in Exercise T2, use a graphing program to do the 
following: 


(a) Plot the surface z = u{x, y) in three-dimensional xyz-space in which z is the temperature at the point (x, y) in the 
square region. 

(b) Plot several isotherms of the temperature distribution (curves in the xy-plane over which the temperature is a 
constant). 

(c) Plot several curves of the temperature as a function of x with y held constant. 

(d) Plot several curves of the temperature as a function of y with x held constant. 


Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 


11.13 

COMPUTED 

TOMOGRAPHY 


In this section we shall see how constructing a cross-sectional view of a human 
body by analyzing X-ray scans leads to an inconsistent linear system. We 
present an iteration technique that provides an “approximate solution” of the 
linear system. 


Linear Systems 
Natural Logarithms 
Euclidean Space R n 


The basic problem of computed tomography is to construct an image of a cross section of the human body using data collected 
from many individual beams of X rays that are passed through the cross section. These data are processed by a computer, and the 
computed cross section is displayed on a video monitor. Figure 11.13.1 is a diagram of General Electric's CT system showing a 
patient prepared to have a cross section of his head scanned by X-ray beams. 



Figure 11.13.1 

Such a system is also known as a CAT scanner, for Computer- Aided Tomography scanner. Figure 11.13.2 shows a typical cross 
section of a human head produced by the system. 



Figure 11.13.2 

The first commercial system of computed tomography for medical use was developed in 1971 by G. N. Hounsfield of EMI, Ltd., 
in England. In 1979, Houndsfield and A. M. Cormack were awarded the Nobel Prize for their pioneering work in the field. As we 
will see in this section, the construction of a cross section, or tomograph, requires the solution of a large linear system of equations. 
Certain algorithms, called (ARTs), can be used to solve these linear systems, whose solutions yield the cross sections in digital 
form. 









Scanning Modes 


Unlike conventional X-ray pictures that are formed by X rays that are projected perpendicular to the plane of the picture, 
tomographs are constructed from thousands of individual, hairline-thin X-ray beams that lie in the plane of the cross section. After 
they pass through the cross section, the intensities of the X-ray beams are measured by an X-ray detector, and these measurements 
are relayed to a computer where they are processed. Figures 11.13.3 and 11.13.4 illustrate two possible modes of scanning the 
cross section: the parallel mode and the fan-beam mode. In the parallel mode a single X-ray source and X-ray detector pair are 
translated across the field of view containing the cross section, and many measurements of the parallel beams are recorded. Then 
the source and detector pair are rotated through a small angle, and another set of measurements is taken. This is repeated until the 
desired number of beam measurements is completed. For example, in the original 1971 machine, 160 parallel measurements were 
taken through 180 angles spaced 1° apart: a total of 160 x 180 = 28,800 beam measurements. Each such scan took approximately 
5-^- mi nutes. 


X-rjy 

dtrlccktr 



source 


Figure 11.13.3 

Parallel mode. 



Figure 11.13.4 

Fan-beam mode. 


In the fan-beam mode of scanning, a single X-ray tube generates a fan of collimated beams whose intensities are measured 
simultaneously by an array of detectors on the other side of the field of view. The X-ray tube and detector array are rotated through 
many angles, and a set of measurements is taken at each angle until the scan is completed. In the General Electric CT system, 
which uses the fan-beam mode, each scan takes 1 second. 


Derivation of Equations 



To see how the cross section is reconstructed from the many individual beam measurements, refer to Figure 11.13.5. Here the field 
of view in which the cross section is situated has been divided into many square pixels (picture elements) numbered 1 through N as 
indicated. It is our desire to determine the X-ray density of each pixel. In the EMI system, 6400 pixels were used, arranged in a 
square 80 x 80 array. The G.E. CT system uses 262,144 pixels ina5l2x512 array, each pixel being about 1 mm on a side. After 
the densities of the pixels are determined by the method we will describe, they are reproduced on a video monitor, with each pixel 
shaded a level of gray proportional to its X-ray density. Because different tissues within the human body have different X-ray 
densities, the video display clearly distinguishes the various tissues and organs within the cross section. 



Figure 11.13.5 

Figure 11.13.6 shows a single pixel with an X-ray beam of roughly the same width as the pixel passing squarely through it. The 
photons constituting the X-ray beam are absorbed by the tissue within the pixel at a rate proportional to the X-ray density of the 
tissue. Quantitatively, the X-ray density of the ;'th pixel is denoted by xj and is defined by 

_ j / number of photons entering the jth pixel \ 

} ^ number of photons leaving the jth pixel ) 

where “In” denotes the natural logarithmic function. Using the logarithm property ]n(a j b) = — ln(£ / a), we also have 

(fraction of photons that pass through \ 

^ the jth pixel without being absorbed J 

If the X-ray beam passes through an entire row of pixels (Figure 11.13.7), then the number of photons leaving one pixel is equal to 
the number of photons entering the next pixel in the row. If the pixels are numbered 1,2,...,«, then the additive property of the 
logarithmic function gives 

/ number of photons entering the fi'st pu:el ) 

I, number of photons leaving the nth pixel j 

Ifraction of photons that pass'' 


Xj= ^ln 


xi 4- X 2 -\ -h Xyi = Inf 


= —In 


through the row of n pixels 
without being absorbed 


( 1 ) 


Thus, to determine the total X-ray density of a row of pixels, we simply sum the individual pixel densities. 
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Figure 11.13.6 
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Figure 11.13.7 


Next, consider the X-ray beam in Figure 11.13.5. By the beam density of the ith beam of a scan, denoted by &, we mean 


f number of photons of the ith beam entering the detector 
wikout the cross section in the field of vew 
number of photons of the ith beam entering the detector 
I with the cross section in the field of view 


= -In 


(fraction of photons of the ith beam that pass through \ 
, the cross section without being absorbed 


( 2 ) 


The numerator in the first expression for is obtained by performing a calibration scan without the cross section in the field of 
view. The resulting detector measurements are stored within the computer's memory. Then a clinical scan is performed with the 
cross section in the field of view, the £ .'s of all the beams constituting the scan are computed, and the values are stored for further 
processing. 


For each beam that passes squarely through a row of pixels, we must have 

'fraction of photons of the beam\ / fraction of photons of the beam 


that pass through the row of 
pixels without being absorbed 




that pass through the cross section 
without being absorbed 


\ 


/ 


Thus, if the ith beam passes squarely through a row of n pixels, then it follows from Equations 1 and 2 that 

x\ + X 2 V- + x n = bj 

In this equation, b i is known from the clinical and calibration measurements, and x\, x 2 , ■ ■ ■, x n are unknown pixel densities that 
must be determined. 


More generally, if the ith beam passes squarely through a row (or column) of pixels with numbers j\, 

+. + x h = bi 

If we set 


then we may write this equation as 



otherwise 


Ji 


j p then we have 


+ arjxi + - -I- a iN x N = bj 


(3) 


We shall refer to Equation 3 as the ith beam equation. 

Referring to Figure 11.13.5, however, we see that the beams of a scan do not necessarily pass through a row or column of pixels 
squarely. Instead, a typical beam passes diagonally through each pixel in its path. There are many ways to take this into account. In 
Figure 11.13.8 we outline three methods of defining the quantities that appear in Equation 3, each of which reduces to our 
previous definition when the beam passes squarely through a row or column of pixels. Reading down the figure, each method is 
more exact than its predecessor, but with successively more computational difficulty. 
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Figure 11.13.8 


Using any one of the three methods to define the a^j s in the ith beam equation, we can write the set of M beam equations in a 
complete scan as 


311*1 

1 12*2 +•' 

"+ = b\ 


<*21*1 

1 322*2 +" 

"+ &2N X N = h 

(4) 

3M1*1 

1 a M2 x 2 T- " 

" T- a MN x N = &M 



In this way we have a linear system of M equations (the M beam equations) in N unknowns (the N pixel densities). 


Depending on the number of beams and pixels used, we may have M > N, M = N, or M < jy. We will consider only the case 
M > N, the so-called over determined case, in which there are more beams in the scan than pixels in the field of view. Because of 
inherent modeling and experimental errors in the problem, we should not expect our linear system to have an exact mathematical 
solution for the pixel densities. In the next section we attempt to find an “approximate” solution to this linear system. 


Algebraic Reconstruction Techniques 


There have been many mathematical algorithms devised to treat the overdetermined linear system 4. The one we will describe 
belongs to the class of so-called Algebraic Reconstruction Techniques (ARTs). This method, which can be traced to an iterative 
technique originally introduced by S. Kaczmarz in 1937, was the one used in the first commercial machine. To introduce this 
technique, consider the following system of three equations in two unknowns: 


Lv. 

*1 

+ *2 = 

2 


Li- 

*1 

- 2 x 2 = 

-2 

(5) 

W 

3*i 

- *2 = 

3 



The lines 1 3 determined by these three equations are plotted in the jqx 2 -plane. As shown in Figure 11.13.9a, the three lines 

do not have a common intersection, and so the three equations do not have an exact solution. However, the points (q, jq) on the 
shaded triangle formed by the three lines are all situated “near” these three lines and can be thought of as constituting 
“approximate” solutions to our system. The following iterative procedure describes a geometric construction for generating points 
on the boundary of that triangular region (Figure 11.13.9/?): 


Algorithm 1 

































Step 0. Choose an arbitrary starting point xq in the A jA v-plane. 


Step 1. Project xo orthogonally onto the first line and call the projection The superscript (1) indicates that this is the 
first of several cycles through the steps. 


Step 2. Project x 0) orthogonally onto the second line and call the projection x 0). 


Step 3. Project x 0) orthogonally onto the third line and call the projection ^.C 1 ). 


Step 4. Take x 0) as the new value of xo and cycle through Steps 1 through 3 again. In the second cycle, label the projected 
points x ©, , x ©; in the third cycle, label the projected points ; and so forth. 







ih) 



{<-) 

Figure 11.13.9 


This algorithm generates three sequences of points 


T 

(1) (2) < 

Lv 

Xj , Xj , X 

Lr 

(1) (?) ( 

X 2 , X 2 , X' 

Lr- 

(1) (?) ( 

X 3 , X 3 , X' 


© 


that lie on the three lines £ 2 , and £ 3 , respectively. It can be shown that as long as the three lines are not all parallel, then the 
first sequence converges to a point x * on the second sequence converges to a point x * on £ 2 , and the third sequence converges 
to a point x * on (Figure 11.13.9c). These three limit points form what is called the limit cycle of the iterative process. It can be 
shown that the limit cycle is independent of the starting point xq- 


We next discuss the specific formulas needed to effect the orthogonal projections in Algorithm 1. First, because the equation of a 








line in xiX2 _s P ace is 


a\xi + a 2 X2 = b 


we can express it in vector form as 


where 


a = b 



and 


x = 


*1 

*2 


The following theorem gives the necessary projection formula (Exercise 5). 


THEOREM 11.13.1 


Orthogonal Projection Formula 

Let L be a line in with equation a ^ and let x * be any point in pf (Figure 11.13.10). Then the orthogonal projection, 
Xp, of s * onto L is given by 


x p =x* + 


(a-* T _ x : 

a 7 a 


a 



Figure 11.13.10 


Using Algorithm 1 


We can use Algorithm 1 to find an approximate solution of the linear system given in 5 and illustrated in Figure 11.13.9. If we 
write the equations of the three lines as 

L\\ afx = 6i 

L 2 '. a2X = &2 

T 

x = *3 

where 



■*r 


V 


r 


3' 

X = 

*2 

a l = 

1 

?, 2 = 

-2 

a 3 = 

- 1 


* 1 = 2 , * 2 = - 2 , £ 3 = 3 

then, using Theorem 11.13.1, we can express the iteration scheme in Algorithm 1 as 



= x 


JP> 

k-l 


~ 3311 2 


k , 


k= 1, 2, 3 


where p = 1 for the first cycle of iterates, p = 2 for the second cycle of iterates, and so forth. After each cycle of iterates [that is. 





















after X (P) is computed], the next cycle of iterates is begun with xq set equal to x &). 

Table 1 gives the numerical results of six cycles of iterations starting with the initial point xq = (1, 3)- 


Table 1 



*1 

*2 

x 0 

1.00000 

3.00000 

x Q 

X 1 

.00000 

2.00000 

x (1) 

x 2 

.40000 

1.20000 

x G) 

x 3 

1.30000 

.90000 

p 

x i 

1.20000 

.80000 

J3 

x 2 

.88000 

1.44000 

(?) 

x 3 

1.42000 

1.26000 

(?) 

X 1 

1.08000 

.92000 

© 

x 2 

.83200 

1.41600 

© 

x 3 

1.40800 

1.22400 

J*> 

x i 

1.09200 

.90800 

(?) 

x 2 

.83680 

1.41840 

(? 

x 3 

1.40920 

1.22760 

P 

1.09080 

.90920 

P 

x 2 

.83632 

1.41816 

x 3 

1.40908 

1.22724 

p 

1.09092 

.90908 

(?) 

x 2 

.83637 

1.41818 















*1 

*2 

m 

1.40909 

1.22728 

x 3 




Using certain techniques that are impractical for large linear systems, we can show the exact values of the points of the limit cycle 
in this example to be 

x* = = (1.09090..., .90909...) 

46 78 


x 2 = ^55’ 55 ) = (- 83636 -’ 1 - 41818 -) 
x 3 * = [QM) = (1.40909..., 1.22727...) 


. 22 ’ 22 , 

It can be seen that the sixth cycle of iterates provides an excellent approximation to the limit cycle. Any one of the three iterates 
can be used as an approximate solution of the linear system. (The large discrepancies in the values of x 'p, x ®, and 

x © are due to the artificial nature of this illustrative example. In practical problems, these discrepancies would be much smaller.) 


To generalize Algorithm 1 so that it applies to an overdetermined system of M equations in N unknowns, 

aiixi + <312*2 H-b aiWXJV = 

021*1 + ^22*2 H-b «2]\r*]V = *2 

ojkf 1*1 + «M2*2 + -+ =&M 

we introduce column vectors x and a, as follows: 


( 6 ) 



*1 


<3,1 


X = 

*2 

= 

an 

rsf 

II 




a iN 



With these vectors, the M equations constituting our linear system 6 can be written in vector form as 

a fx = bj, i = 

Each of these M equations defines what is called a hyperplane in the A-dimensional Euclidean space j n general these M 
hyperplanes have no common intersection, and so we seek instead some point in ^ that is reasonably “close” to all of them. Such 
a point will constitute an approximate solution of the linear system, and its N entries will determine approximate pixel densities 
with which to form the desired cross section. 

As in the two-dimensional case, we will introduce an iterative process that generates cycles of successive orthogonal projections 
onto the M hyperplanes beginning with some arbitrary initial point in Our notation for these successive iterates is 


>)_ 

— 


I the ierate lying on the kth hyperplane \ 
l ,generated daring ihepih cycle of iterations , 


The algorithm is as follows: 

Algorithm 2 


Step 0. Choose any point in and label it xg. 


Step 1. For the first cycle of iterates, set p = 1. 











Step 2. For fc = \, 2, M, compute 






Step 3. Set(P+V)_ (p) m 
*o — a m 

Step 4. Increase the cycle number p by 1 and return to Step 2. 


In Step 2 the iterate x (p) is called the orthogonal projection of onto the hyperplane a Consequently, as in the 
h h — 1 ™ 

two-dimensional case, this algorithm determines a sequence of orthogonal projections from one hyperplane onto the next in which 
we cycle back to the first hyperplane after each projection onto the last hyperplane. 


It can be shown that if the vectors a i, <n, • • •> a m s P an R N > then the iterates .X 1 ), C 2 ), (3), ... lying on the Mth hyperplane will 

A M A M A M 

converge to a point on ^ at hyperplane which does not depend on the choice of the initial point jeq. In computed tomography, 
one of the iterates 0 3 ) for p sufficiently large is taken as an approximate solution of the linear system for the pixel densities. 


Note that for the center-of-pixel method, the scalar quantity -,7^, appearing in the equation in Step 2 of the algorithm is simply the 
number of pixels in which the &th beam passes through the center. Similarly, note that the scalar quantity 




T (p) 
k x k -1 


in that same equation can be interpreted as the excess kth beam density that results if the pixel densities are set equal to the entries 

of GO . This provides the following interpretation of our ART iteration scheme for the center-of-pixel method: Generate the pixel 
k —1 


densities of each iterate by distributing the excess beam density of successive beams in the scan evenly among those pixels in 
which the beam passes through the center. When the last beam in the scan has been reached, return to the first beam and continue. 


Using Algorithm 2 


We can use Algorithm 2 to find the unknown pixel densities of the 9 pixels arranged in the 3x3 array illustrated in Figure 
11.13.11. These 9 pixels are scanned using the parallel mode with 12 beams whose measured beam densities are indicated in the 
figure. We choose the center-of-pixel method to set up the 12 beam equations. (In Exercises 7 and 8, the reader is asked to set up 
the beam equations using the center line and area methods.) As the reader can verify, the beam equations are 


x 7 

+ xg 

1 = 13.00 

X3 

-1- x$ +xg = 

18.00 

x 4 

+ x; 

+ xfi = 15.00 

X2 

+ x 5 + xg = 

12.00 

*1 

+ x 2 

-\-X2= 8.00 

xi 

-1- X4+ x-] = 

6.00 

*6 

+ Xg 

-h ^9 = 14.79 

x 2 

+ X3 + x,5 = 

10.51 

x 3 

+ XJ 

+ *7 = 14.31 

xi 

-1- X5 + *9 = 

16.13 

x 1 

+ x 2 

-f 7:4 = 3.81 

x 4 

-1- x 7 + xg = 

7.04 


Table 2 illustrates the results of the iteration scheme starting with an initial iterate X q = 0- The table gives the values of each of the 

first cycle of iterates, 0) through 0), but thereafter gives the iterates (p) only for various values of p. The iterates x Cp) start 
112 12 12 

repeating to two decimal places for p > 45, and so we take the entries of as approximate values of the 9 pixel densities. 
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Table 2 



INiel Ik’iisiiivs 


x i 

X 2 

X \ 

x i 

*5 

*6 

X 1 


X 9 

X n 

.00 

.00 

.00 

.00 

.00 

.00 

.00 

.00 

(XI 


.00 

.00 

.go 

.00 

.00 

.00 

4.33 

4.33 

4.33 

v Lll 

*1 

.00 

.00 

.0)3 

5.00 

5.00 

5.00 

4.33 

4.3.3 

4.33 


2.67 

2.67 

2.67 

5.00 

5.00 

5-00 

4.33 

4.33 

4.33 

x‘" 

2.67 

2.67 

2.67 

5.00 

5.1X1 

5.37 

4.33 

4.71 

4.7! 

x‘” 

2.67 

2.67 

3.44 

5.00 

5.77 

5.37 

5.10 

4.71 

4.71 


.40 

.40 

3.44 

2.83 

5.77 

5.37 

5.10 

4.71 

4.71 

x' lj 

.40 

.40 

4.03 

2,83 

5.77 

6.87 

5.10 

4.71 

6.20 

ill 

.40 

.84 

4.93 

2.83 

6.11 

6.87 

5.10 

5.05 

6,20 

*«:■> 

-.31 

.84 

4.93 

2.02 

6.11 

6.87 

4,30 

5.05 

6.20 

x' 11 
lu 

-.31 

.13 

4.22 

2.02 

6,11 

6.16 

4.30 

5.05 

6.20 

x o> 

*11 

1.06 

.13 

4.22 

2.02 

7.49 

6.16 

4.30 

5.05 

7.58 

*!2 

1.06 

.13 

4.22 

.58 

7.49 

6. !6 

2.85 

3.61 

7.58 

v^ 1 

*11 

2,03 

,60 

4,42 

1,34 

7,49 

5,39 

2,65 

3.04 

6.6] 

*11 

1.78 

.51 

4.52 

1.26 

7.49 

5.48 

2.56 

3.22 

6.86 


1,82 

.52 

4.62 

1,37 

7.49 

5,37 

2.45 

.3,22 

6.82 

*12 

1.70 

.49 

4,71 

t.4.3 

7.49 

5.31 

2.37 

3.25 

6.85 


t.68 

.44 

53)3 

1.70 

7.49 

5.03 

2.04 

3.29 

6.96 

*12 

1.40 

.48 

5.29 

2.1X) 

7.49 

4,7.3 

1.79 

.3.25 

7.15 

v ^ai 

*12 

1.38 

.55 

5.34 

2.11 

7.49 

4.62 

1.74 

3.19 

7.26 

V r-Hip 

*12 

1.33 

.59 

5.33 

2,14 

7.49 

4.59 

1,75 

.3.15 

7.31 

’'ii 

1.32 

.60 

5.32 

2,15 

7,49 

4.59 

1.76 

.3,14 

7.32 


We close this section by noting that the field of computed tomography is presently a very active research area. In fact, the ART 
scheme discussed here has been replaced in commercial systems by more sophisticated techniques that are faster and provide a 
more accurate view of the cross section. However, all the new techniques address the same basic mathematical problem: finding a 
good approximate solution of a large overdetermined inconsistent linear system of equations. 


Exercise Set 11.13 
































































Click here for Just Ask! 


1 . 


( a ) Setting x (p) = (x^ s * low that Ae three projection equations 


(P) „(p) 


x t =x 


■Jc-l 


(£fc-44-i) 


for the three lines in Equation 5 can be written as 




k = 1, 2, 3 


k= 1 : 


_ _ 1 _ro 

f ll - 2 V 


,<P) r cp), 
— *02 J 


l 01 


>)_i r2 (p) , (p), 

*12 — o — *01 + *02 J 


£= 2 : 


£=3: 


4 ) = I [-2 + < ) + 24 ) ] 

z®-1[4 + 2»® + *®] 


_(P)_ 1 ro 

-To [9 

_Cp) 


,Cp) 

r 21 


3 * 2 ? 1 


(p) 


Y 32 - IQ t _ 3 + J *21 


44 


where (4 +1) . 4 +1) ) = (4 : '4 } ) for ^ =1 ' 2 > ■■■■ 


(b) Show that the three pairs of equations in part (a) can be combined to produce 


(P)_ 1 roo . JP-l) CP-l)n 

31 “20 L " + 31 32 


,<p) 


l 32 — 20 ^ h 


(P-1) 


3x 


te-14 

32 J 


^ = 1 , 2 ,... 


where fl <P) ^ _ , CD OX a). 

(*31 ’ *32 J ~ (*01 - *02 } ~ *0 

Note Using this pair of equations, we can perform one complete cycle of three orthogonal projections in a single step. 

( c ) Because X 0 J ) tends to the limit point s * as p - * oo, the equations in part (b) become 


*31 — 2(4 8 ^*31 ~ X32 ^ 


* 1 * 
*32 = 20"[ 24 f 3x 


31 


■34] 


as p —♦ oo . Solve this linear system for x * = (4i> 4)' 

Note The simplifications of the ART formulas described in this exercise are impractical for the large linear systems that 
arise in realistic computed tomography problems. 

Use the result of Exercise 1(b) to find (1), (9, ..., © to five decimal places in Example 1 using the following initial points: 
2, 3 3 3 


(a) x 0 = ( 0 , 0 ) 


(b) x 0 = (1, 1) 






(c) x 0 = (148, -15) 


3. 


(a) Show directly that the points of the limit cycle in Example 1, 



form a triangle whose vertices lie on the lines l 2 , and l 3 and 
to these lines (Figure 11.13.9c). 


whose sides are perpendicular 


(b) Using the equations derived in Exercise 1(a), show that if x^ 




Note Either part of this exercise shows that successive orthogonal projections of any point on 
the limit cycle will move around the limit cycle indefinitely. 

The following three lines in the x^-plane, 


4. 


Ly. x 2 =l 

Ly. xi—*2 = 2 
Ly. *1—x 2 = 0 


do not have a common intersection. Draw an accurate sketch of the three lines and graphically performseveral cycles of the 
orthogonal projections described in Algorithm 1, beginning with the initial point xq = (0, 0)- On the basis of your sketch, 
determine the three points of the limit cycle. 

Prove Theorem 11.13.1 by verifying that 

5. 

(a) the point \ p as defined in the theorem lies on the line a ^ (i.e., ^ p — £,); 

(b) the vector x _ x* is orthogonal to the line a (that is, — x * is parallel to a). 


As stated in the text, the iterates ..C 1 ), ,.(9, ..C 3 ), ... defined in Algorithm 2 will converge to a unique limit point Y * if the 

6. X M X M X M 

vectors a i, < 12 , • • •» a M s P an R^- Show that if this is the case and if the center-of-pixel method is used, then the center of each of 
the N pixels in the field of view is crossed by at least one of the M beams in the scan. 

Construct the 12 beam equations in Example 2 using the center line method. Assume that the distance between the center lines 

7- of adjacent beams is equal to the width of a single pixel. 

Construct the 12 beam equations in Example 2 using the area method. Assume that the width of each beam is equal to the width 

8- of a single pixel and that the distance between the center lines of adjacent beams is also equal to the width of a single pixel. 



Section 11.13 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Tl. 


Given the set of equations 


a k x + b k y = Ck 

for £=1,2, 3, ...,« (withn n > 2), let us consider the following algorithm for obtaining an approximate solution to the 
system. 

1, Solve all possible pairs of equations 


ajX 4 bjV = Cj and ajx 4 bjy = Cj 

for /', j = l, 2, 3, ..., n and j < j for their unique solutions. This leads to 


1 


-«(«-!) 


solutions, which we label as 
for /', j=\, 2, 3, ..., n and i< j. 




2. Construct the geometric center of these points defined by 


(xc.yc) = 




h —1 n 

£ £ x,. 


H —1 H 


£. £ / Ki 


\»C»“ 1) i=lj=i4l i=lj=i4l 

and use this as the approximate solution to the original system. 

Use this algorithm to approximate the solution to the system 

*+ 7 = 2 
x — 2 y = - 2 
3x - 7 = 3 

and compare your results to those in this section. 


T2. 


(For Readers Who Have Studied Calculus) 

Given the set of equations 


^hx + b k y=c k 

for £ = 1, 2, 3, ..., n (with n > 2), let us consider the following least squares algorithm for obtaining an approximate solutk 
(x ,y ) to the system. Given a point ( Qi gj and the line a ^x 4 £, v = c,» the distance from this point to the line is given by 

|fljQ:4 bjft — cA 


l 


of 4 bf 


If we define a function j ( Xi y) by 
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FRACTALS 


In this section we shall use certain classes of linear transformations to describe 
and generate intricate sets in the Euclidean plane. These sets, called fractals, 
are currently the focus of much mathematical and scientific research. 


Geometry of Linear Operators on p} (Section 9.2) 

Euclidean Space R n 

Natural Logarithms 

Intuitive Understanding of Limits 


Fractals in the Euclidean Plane 

At the end of the nineteenth century and the beginning of the twentieth century, various bizarre and wild sets of points in the 
Euclidean plane began appearing in mathematics. Although they were initially mathematical curiosities, these sets, called fractals, 
are rapidly growing in importance. It is now recognized that they reveal a regularity in physical and biological phenomena 
previously dismissed as “random,” “noisy,” or “chaotic.” For example, fractals are all around us in the shapes of clouds, 
mountains, coastlines, trees, and ferns. 

In this section we give a brief description of certain types of fractals in the Euclidean plane p\ Much of this description is an 
outgrowth of the work of two mathematicians, Benoit B. Mandelbrot and Michael Barnsley, who are both active researchers in the 
field. 

Self-Similar Sets 

To begin our study of fractals, we need to introduce some terminology about sets in p}. We shall call a set in p} bounded if it can 
be enclosed by a suitably large circle (Figure 11.14.1) and closed if it contains all of its boundary points (Figure 11.14.2). Two sets 
in p} will be called congruent if they can be made to coincide exactly by translating and rotating them appropriately within p A 
(Figure 11.14.3). We will also rely on the reader's intuitive concept of overlapping and nonoverlapping sets, as illustrated in 
Figure 11.14.4. 





(a) Set enclosed by a circle 




[ n bounded 


-V 

(b) This sel cannot be 
enclosed by any circle. 

Figure 11.14.1 



Figure 11.14.2 

The boundary points (solid color) lie in the set. 



Figure 11.14.3 













V 



(«) Over I tipping set s 



ib) Nonovertapping sets 

Figure 11.14.4 

If f p} _* p} is the linear operator that scales by a factor of s (see Table 8 of Section 4.2), and if Q is a set in p2, then the set 

T(Q) (the set of images of points in Q under T) is called a dilation of the set Q if s > ] and a contraction of Q if Q < s < ] (Figure 
11.14.5). In either case we say that T(Q) is the set Q scaled by the factors. 



Figure 11.14.5 

A contraction of Q. 


The types of fractals we shall consider first are called self-similar. In general, we define a self-similar set in p} as follows: 


DEFINITION 


A closed and bounded subset of the Euclidean plane p} is said to be self-similar if it can be expressed in the form 

S=Si \JS 2 uS 3 u~uS k 

where S'j, are nonoverlapping sets, each of which is congruent to S scaled by the same factor s (0 < s < !)• 


( 1 ) 


If S is a self-similar set, then 1 is sometimes called a decomposition of S into nonoverlapping congruent sets. 































Line Segment 


A line segment in (Figure 11.14.6a) can be expressed as the union of two nonoverlapping congruent line segments (Figure 
11.14.6Z?). In Figure 11.14.6Z? we have separated the two li ne segments slightly so that they can be seen more easily. Each of these 
two smaller line segments is congruent to the original line segment scaled by a factor of -i. Hence, a line segment is a self-similar 

set with k = 2 and s = 


6/1 


ih) 

Figure 11.14.6 


♦ 


Square 


A square (Figure 11.14.7a) can be expressed as the union of four nonoverlapping congruent squares (Figure 11.14.7Z?), where we 
have again separated the smaller squares slightly. Each of the four smaller squares is congruent to the original square scaled by a 
factor of -1. Hence, a square is a self-similar set with k = 4 and s = 



U') 



{b\ 

Figure 11.14.7 


Sierpinski Carpet 


The set suggested by Figure 11.14.8a, the Sierpinski “carpet,” was first described by the Polish mathematician Waclaw Sierpinski 











(1882-1969). It can be expressed as the union of eight nonoverlapping congruent subsets (Figure 11.14.8Z?), each of which is 
congruent to the original set scaled by a factor of -i. Hence, it is a self-similar set with k = 8 and s = \ ) - Note that the intricate 

square-within-a-square pattern continues forever on a smaller and smaller scale (although this can only be suggested in a figure 
such as the one shown). 




Sierpinski Triangle 


Figure 11.14.9a illustrates another set described by Sierpinski. It is a self-similar set with k = 3 and s = (Figure 11.14.9/?). As 
with the Sierpinski carpet, the intricate triangle-within-a-triangle pattern continues forever on a smaller and smaller scale. 
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Figure 11.14.9 
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The Sierpinski carpet and triangle have a more intricate structure than the line segment and the square in that they exhibit a pattern 
that is repeated indefinitely. This difference will be explored later in this section. 


Topological Dimension of a Set 

In Section 5.4 we defined the dimension of a subspace of a vector space to be the number of vectors in a basis, and we found that 





































































definition to coincide with our intuitive sense of dimension. For example, the origin of pf is zero-dimensional, lines through the 
origin are one-dimensional, and p2 itself is two-dimensional. This definition of dimension is a special case of a more general 
concept called topological dimension, which is applicable to sets in p n that are not necessarily subspaces. A precise definition of 
this concept is studied in a branch of mathematics called topology. Although that definition is beyond the scope of this text, we can 
state informally that 


a point in p 2 has topological dimension zero; 


a curve in p 2 has topological dimension one; 


a region in p 2 has topological dimension two. 

It can be proved that the topological dimension of a set in p n must be an integer between 0 and n, inclusive. In this text we shall 
denote the topological dimension of a set S by dj (£)• 


EXAMPLE 5 Topological Dimensions of Sets 


Table 1 gives the topological dimensions of the sets studied in our earlier examples. The first two results in this table are intuitively 
obvious; however, the last two are not. Informally stated, the Sierpinski carpet and triangle both contain so many “holes” that those 
sets resemble web-like networks of lines rather than regions. Hence they have topological dimension one. The proofs are quite 
difficult. 


Table 1 

Set 5 

d T (Z-) 

Line segment 

1 

Square 

2 

Sierpinski carpet 

1 

Sierpinski triangle 

1 


Hausdorff Dimension of a Self-Similar Set 

In 1919 the German mathematician Felix Hausdorff (1868-1942) gave an alternative definition for the dimension of an arbitrary 
set in R n . His definition is quite complicated, but for a self-similar set, it reduces to something rather simple: 


DEFINITION 


The Hausdorff dimension of a self-similar set S of form 1 is denoted by d^(S) and is defined by 















In this definition, “ In” denotes the natural logarithm function. Equation 2 can also be expressed as 


,^agS) 1 

* =k (3) 

in which the Hausdorff dimension d^{S) appears as an exponent. Formula 3 is more helpful for interpreting the concept of 
Hausdorff dimension; it states, for example, that if you scale a self-similar set by a factor of s = then its area (or, more properly, 

its measure ) decreases by a factor of [2. j . Thus, scaling a line segment by a factor of ± reduces its measure (length) by a 


and scaling a square region by a factor of reduces its measure (area) by a factor of j = i. 


factor of P-'i = i, 

UJ 2 

Before proceeding to some examples, we should note a few facts about the Hausdorff dimension of a set: 


The topological dimension and Hausdorff dimension of a set need not be the same. 


The Hausdorff dimension of a set need not be an integer. 


The topological dimension of a set is less than or equal to its Hausdorff dimension; that is, dj(S) < d #(£)• 


EXAMPLE 6 Hausdorff Dimensions of Sets 

Table 2 lists the Hausdorff dimensions of the sets studied in our earlier examples. 

Table 2 


Set 5 

s 

k 





ln(l/s) 

Line segment 

1 

2 

2 

In2/ln2 = l 

Square 

1 

2 

4 

In 4 / In 2 = 2 

Sierpinski carpet 

1 

3 

8 

In8/ln3 = 1.892... 

Sierpinski triangle 

1 

2 

3 

In3/ln2 = 1.584... 


Fractals 


Comparing Tables 1 and 2, we see that the Hausdorff and topological dimensions are equal for both line segment and square but 
are unequal for the Sierpinski carpet and triangle. In 1977 Benoit B. Mandelbrot suggested that sets for which the topological and 














Hausdorff dimensions differ must be quite complicated (as Hausdorff had earlier suggested in 1919). Mandelbrot proposed calling 
such sets fractals, and he offered the following definition. 


DEFINITION 


A fractal is a subset of a Euclidean space whose Hausdorff dimension and topological dimension are not equal. 


Mandelbrot has suggested that this definition is rather limited and probably will be replaced in the future. But in the meantime, it 
remains the formal definition of a fractal. According to this definition, the Sierpinski carpet and Sierpinski triangle are fractals, 
whereas the line segment and square are not. 

It follows from the preceding definition that a set whose Hausdorff dimension is not an integer must be a fractal (why?). However, 
we will see later that the converse is not true; that is, it is possible for a fractal to have an integer Hausdorff dimension. 

Similitudes 

We shall now show how some techniques from linear algebra can be used to generate fractals. This linear algebra approach also 
leads to algorithms that can be exploited to draw fractals on a computer. We begin with a definition. 


DEFINITION 


A similitude with scale factor s is a mapping of p} into pf of the form 


= s 


cos # — sin# 

sin# cos# 


+ 


where s, 0, e, and/are scalars. 


Geometrically, a similitude is a composition of three simpler mappings: a scaling by a factor of s, a rotation about the origin 
through an angle 0, and a translation (e units in the x-direction and/units in the y-direction). Figure 11.14.10 illustrates the effect 
of a similitude on the unit square U. 





















(/>) Unit sniiinv 
after similitude 


Figure 11.14.10 


For our application to fractals, we shall need only similitudes that are contractions, by which we mean that the scale factor s is 
restricted to the range Q < s < ]. Consequently, when we refer to similitudes we shall always mean similitudes subject to this 
restriction. 

Similitudes are important in the study of fractals because of the following fact: 


Iff. pf _j pf is a similitude with scale factor s and ifS is a closed and bounded set in pf, then the image T(S) of the set S 

under T is congruent to S scaled by s. 


Recall from the definition of a self-similar set in pf that a closed and bounded set S in pf is self-similar if it can be expressed in 
the form 

S = Si U S 2 U S 3 U - U S k 

where ff, ^ Sy ..., S k are nonoverlapping sets each of which is congruent to S scaled by the same factor sf O < s < 1) [see 1], In 
the following examples, we will find similitudes that produce the sets Sfr from S for the line segment, square, 

Sierpinski carpet, and Sierpinski triangle. 


Line Segment 


We shall take as our line segment the line segment S connecting the points (0, 0) and (1, 0) in the xy-plane (Figure 11.14.1 la). 
Consider the two similitudes 
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( 4 ) 


both of which have s = and Q = 0. In Figure 11.14.11^ we show how these two similitudes map the unit square U. The 

similitude T l maps U onto the smaller square T\ (U), and the similitude t 2 maps U onto the smaller square T 2 (U)- At the 
time, 7*j maps the line segment S onto the smaller line segment T\ {S), and t 2 maps S onto the smaller nonoverlapping li ne 
segment 7^ (S). The union of these two smaller nonoverlapping line segments is precisely the original line segment S', that is, 
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Figure 11.14.11 


Square 


Let us consider the unit square U in the xy-plane (Figure 11.14.12a) and the following four similitudes, all having s = 4- and 9=0 
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Figure 11.14.12 

The images of the unit square U under these four similitudes are the four squares shown in Figure 11.14.12&. Thus, 

U=T l (U)uT 2 (U)uT 3 (U)uT 4 (U) 

is a decomposition of U into four nonoverlapping squares that are congruent to U scaled by the same scale factor js = y j. 


(7) 


Sierpinski Carpet 


Let us consider a Sierpinski carpet S over the unit square U of the xy-plane (Figure 11.14.13a) and the following eight similitudes, 
all having s = y and Q = Q: 
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( 8 ) 


where the eight values of 


Ji 


are 


The images of S under these eight similitudes are the eight sets shown in Figure 11.14.13&. Thus, 

S = Ti (£) U T 2 OS) J T 3 (£)•■■ U T % (£) 

is a decomposition of S into eight nonoverlapping sets that are congruent to S scaled by the same scale factor = y j. 


( 9 ) 
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Figure 11.14.13 



EXAMPLE 10 Sierpinski Triangle 


Let us consider a Sierpinski triangle S fitted inside the unit square U of the xy-plane, as shown in Figure 11.14.14a, and the 


following three similitudes, all having s = and @ = 0 : 
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( 10 ) 


The images of S under these three similitudes are the three sets in Figure 11.14.14 b. Thus, 

S=T] (S) u Ti($) u TVS) 

is a decomposition of S into three nonoverlapping sets that are congruent to S scaled by the same scale factor j. 
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Figure 11.14.14 


In the preceding examples we started with a specific set S and showed that it was self-similar by finding similitudes Jj, j 2 , T 3 , ..., 
T k with the same scale factor such that ^ (5), ^(S), ^(S). •••, T k (2) were nonoverlapping sets and such that 













































S=T 1 (S)uT 2 (S)uT 3 (S)u...uT k (S) 


( 12 ) 


The following theorem addresses the converse problem of determining a self-similar set from a collection of similitudes. 


THEOREM 11.14.1 


If Tv Ty Ty •••> T k > are contracting similitudes with the same scale factor, then there is a unique nonempty closed and 
bounded set S in the Euclidean plane such that 

S=T l (S)L>T 2 (S)L>T 3 (X)U...uT k (X) 

Furthermore, if the sets T\(S), T 2 (S), T 3 (S), ..., T k (£) are nonoverlapping, then S is self-similar. 


Algorithms for Generating Fractals 

In general, there is no simple way to obtain the set S in the preceding theorem directly. We now describe an iterative procedure that 
will determine S from the similitudes that define it. We first give an example of the procedure and then give an algorithm for the 
general case. 


Sierpinski Carpet 


Figure 11.14.15 shows the unit square region in the xy-plane, which will serve as an “initial” set for an iterative procedure for 
the construction of the Sierpinski carpet. The set in the figure is the result of mapping £q with each of the eight similitudes 
Tj(i = 1, 2,..., 8) in 8 that determine the Sierpinski carpet. It consists of eight square regions, each of side length j, surrounding 

an empty middle square. We next apply the eight similitudes to and arrive at the set s 2 - Similarly, applying the eight similitudes 
to $2 results in the set Sy It we continue this process indefinitely, the sequence of sets Sp Sy Sy ■■■ will “converge” to a set S, 
which is the Sierpinski carpet. 



Figure 11.14.15 
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Remark Although we should properly give a definition of what it means for a sequence of sets to “converge” to a given set, an 
intuitive interpretation will suffice in this introductory treatment. 


Although we started in Figure 11.14.15 with the unit square region to arrive at the Sierpinski carpet, we could have started with 
any nonempty set ^q. The only restriction is that the set £q be closed and bounded. For example, if we start with the particular set 
£q shown in Figure 11.14.16, then is the set obtained by applying each of the eight similitudes in 8. Applying the eight 
similitudes to results in the set S- 2 - As before, applying the eight similitudes indefinitely yields the Sierpinski carpet S as the 
limiting set. 



Figure 11.14.16 


The general algorithm illustrated in the preceding example is as follows: Let 7 ^, 7 ^, ..., T k be contracting similitudes with the 
same scale factor, and for an arbitrary set Q in p}, define the set J{Q) by 

J(Q) = Ti (0 U r 2 (0 U r 3 (0 U ... U T k (Q) 

The following algorithm generates a sequence of sets Sq, 3^, ... that converges to the set S in Theorem 11.14.1. 


Algorithm 1 

Step 0. Choose an arbitrary nonempty closed and bounded set g Q in 
Step 1. Compute s 1 = J(S 0 ) ■ 


Step 2. Compute S 2 = J(Sy)- 








































































































Step 3 . Compute Sj = JiS-i)- 


Step n. Compute s» = J(S„-i)- 


Sierpinski Triangle 


Let us construct the Sierpinski triangle determined by the three similitudes given in 10. The corresponding set mapping is 
,7(0 = T\ (Q) U T'JyQ) U ?3(0- Figure 11.14.17 shows an arbitrary closed and bounded set the first four iterates Sy 

gq and the limiting set S (the Sierpinski triangle). 
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Figure 11.14.17 


Using Algorithm 1 
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Consider the following two similitudes: 















The actions of these two similitudes on the unit square U are illustrated in Figure 11.14.18. Here, the rotation angle 0 is a 
parameter that we shall vary to generate different self-similar sets. The self-similar sets determined by these two similitudes are 
shown in Figure 11.14.19 for various values of 0 . For simplicity, we have not drawn the xy-axes, but in each case the origin is the 
lower left point of the set. These sets were generated on a computer using Algorithm 1 for the various values of 9 . Because k=2 
and s = -i, it follows from 2 that the Hausdorff dimension of these sets for any value of 0 is 1. It can be shown that the topological 

dimension of these sets is 1 for Q = 0 and 0 for all other values of 0 . It follows that the self-similar set for 9 = 0 is not a fractal [it is 
the straight line segment from (0, 0) to (.6, .6)], while the self-similar sets for all other values of 0 are fractals. In particular, they 
are examples of fractals with integer Hausdorff dimension. 
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Figure 11.14.18 




Figure 11.14.19 
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A Monte Carlo Approach 

The set-mapping approach of constructing self-similar sets described in Algorithm 1 is rather time-consuming on a computer 
because the similitudes involved must be applied to each of the many computer screen pixels in the successive iterated sets. In 
1985 Michael Barnsley described an alternative, more practical method of generating a self similar set defined through its 
similitudes. It is a so-called Monte Carlo method that takes advantage of probability theory. Barnsley refers to it as the Random 
Iteration Algorithm. 


Let Jj, T 2 , Ty • ••■> T% be contracting similitudes with the same scale factor. The following algorithm generates a sequence of 
points 
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that collectively converge to the set S in Theorem 11.14.1. 


Algorithm 2 





















Step 0. Choose an arbitrary point 


*0 

70 


in S. 


Step 1. Choose one of the k similitudes at random, say 7 ^, and compute 
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y i 




7o 


Step 2. Choose one of the k similitudes at random, say TV-,, and compute 
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72 




*1 

71 


Step n. Choose one of the k similitudes at random, say , and compute 


7m 




— 1 
7m-1 


On a computer screen the pixels corresponding to the points generated by this algorithm will fill out the pixel representation of the 
limiting set S. 


Figure 11.14.20 shows four stages of the Random Iteration Algorithm that generate the Sierpinski carpet, starting with the initial 
~ 0 ' 


point 
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Figure 11.14.20 


Remark Although Step 0 in the preceding algorithm requires the selection of an initial point in the set S, which may not be known 
in advance, this is not a serious problem. 


In practice, one can usually start with any point in 7 2 and after a few iterations (say ten or so), the point generated will be 
sufficiently close to S that the algorithm will work correctly from that point on. 









































More General Fractals 


So far, we have discussed fractals that are self-similar sets according to the definition of a self-similar set in p}. However, 
Theorem 11.14.1 remains true if the similitudes 7 ^, 7 ^, ..., are replaced by more general transformations, called contracting 
affine transformations. An affine transformation is defined as follows: 


DEFINITION 


An affine transformation is a mapping oi pf 2 into p 2 of the form 

T 

where a, b, c, d, e, and/are scalars. 


a b 
c d 


Figure 11.14.21 shows how an affine transformation maps the unit square U onto a parallelogram T(U)- An affine transformation 
is said to be contracting if the Euclidean distance between any two points in the plane is strictly decreased after the two points are 
mapped by the transformation. It can be shown that any k contracting affine transformations T v ... determine a unique 
closed and bounded set S satisfying the equation 

S = T\ iffi) U T 2 (£) U T 3 (£) U... U T k (£) 


Equation 13 has the same form as Equation 12, which we used to find self-similar sets. Although Equation 13, which uses 
contracting affine transformations, does not determine a self-similar set S, the set it does determine has many of the features of 
self-similar sets. For example, Figure 11.14.22 shows how a set in the plane resembling a fern (an example made famous by 
Barnsley) can be generated through four contracting affine transformations. Note that the middle fern is the slightly overlapping 
union of the four smaller affine-image ferns surrounding it. Note also how 7 ^, because the determinant of its matrix part is zero, 
maps the entire fern onto the small straight li ne segment between the points (.50, 0) and (.50, .16). Figure 11.14.22 contains a 
wealth of information and should be studied carefully. 
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Figure 11.14.21 
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Figure 11.14.22 


Michael Barnsley is actively pursuing an application of the above theory to the field of data compression and transmission. The 
fern, for example, is completely determined by the four affine transformations T\, 7/ 7/, 7 4 - These four transformations, in turn, 
are determined by the 24 numbers given in Figure 11.14.22 defining their corresponding values of a, b, c, d, e, and/. In other 
words, these 24 numbers completely encode the picture of the fern. Storing these 24 numbers in a computer requires considerably 
less memory space than storing a pixel-by-pixel description of the fern. In principle, any picture represented by a pixel map on a 
computer screen can be described through a finite number of affine transformations, although it is not easy to determine which 
transformations to use. Nevertheless, once encoded, the affine transformations generally require several orders of magnitude less 
computer memory than a pixel-by-pixel description of the pixel map. 

Further Readings 

Readers interested in learning more about fractals are referred to the following books, the first of which elaborates on the linear 
transformation approach of this section. 


1. Michael Barnsley, Fractals Everywhere (New York: Academic Press, 1993). 

BENOIT B. Mandelbrot, The Fractal Geometry of Nature (New York: W. H. Freeman, 1982). 
Heinz-OTTO Peitgen and P. H. Richter, The Beauty of Fractals (New York: Springer-Verlag, 1986). 











































4, Heinz-OTTO Peitgen and Dietmar Saupe, The Science of Fractal Images (New York: Springer-Verlag, 1988). 


Exercise Set 11.14 



Click here for Just Ask! 


The self-similar set in Figure Ex-1 has the sizes indicated. Given that its lower left comer is situated at the origin of the xy 
1* -plane, find the similitudes that determine the set. What is its Hausdorff dimension? Is it a fractal? 
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Figure Ex-1 


Find the Hausdorff dimension of the self-similar set shown in Figure Ex-2. Use a ruler to measure the figure and determine an 
2- approximate value of the scale factor s. What are the rotation angles of the similitudes determining this set? 



Figure Ex-2 


For each of the self-similar sets in the accompanying figure, find: (i) the scale factor s of the similitudes describing the set; (ii) 
3* the rotation angles 0 of all similitudes describing the set (all rotation angles are multiples of 90°; and (iii) the Hausdorff 
dimension of the set. Which of the sets are fractals and why? 












Show that of the four affine transformations shown in Figure 11.14.22, only the transformation 7 2 is a similitude. Deter mi ne its 
4* scale factor 5 and rotation angle ff. 


Find the coordinates of the tip of the fern in Figure 11.14.22. 

5. 


Hint The transformation maps the tip of the fern to itself. 


The square in Figure 11.14.7a was expressed as the union of 4 nonoverlapping squares as in Figure ll.14.7Zx Suppose that it is 
6* expressed instead as the union of 16 nonoverlapping squares. Verify that its Hausdorff dimension is still two, as determined by 
Equation 2. 


Show that the four similitudes 
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express the unit square as the union of four overlapping squares. Evaluate the right-hand side of Equation 2 for the values of 
and s determined by these similitudes, and show that the result is not the correct value of the Hausdorff dimension of the unit 





































































square. 


Note This exercise shows the necessity of the nonoverlapping condition in the definition of a self-similar set and its Hausdorff 
dimension. 

All of the results in this section can be extended to R n . Compute the Hausdorff dimension of the unit cube in R s (see Figure 
Ex-8). Given that the topological dimension of the unit cube is three, determine whether it is a fractal. 



Figure Ex-8 


Hint Express the unit cube as the union of eight smaller congruent nonoverlapping cubes. 


The set in r} in the accompanying figure is called the Menger sponge. It is a self-similar set obtained by drilling out certain 
9* square holes from the unit cube. Note that each face of the Menger sponge is a Sierpinski carpet and that the holes in the 
Sierpinski carpet now run all the way through the Menger sponge. Determine the values of k and s for the Menger sponge and 
find its Hausdorff dimension. Is the Menger sponge a fractal? 



Figure Ex-9 

The two similitudes 

2 " 

3 

0 _ 

determine a fractal known as the Cantor set. Starting with the unit square region U as an initial set, sketch the first four sets 
that Algorithm 1 determines. Also, find the Hausdorff dimension of the Cantor set. (This famous set was the first example that 
Hausdorff gave in his 1919 paper of a set whose Hausdorff dimension is not equal to its topological dimension.) 
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Compute the areas of the sets gg, £ 3 , anc * £4 i * 1 Figure 11.14.15. 


Section 11.14 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Use similitudes of the form 
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to show that the Menger sponge (see Exercise 9) is the set S satisfying 

S= l£j TAZ) 

for appropriately chosen similitudes (for i = 1, 2, 3,..., 20). Determine these similitudes by determining the collection of 
3x1 matrices 


a, 

bi 


for z = 1, 2, 3,..., 20 


T2. 


Generalize the ideas involved in the Cantor set (in j? 1 ), the Sierpinski carpet (in j? 2 ), and the Menger sponge (in r}) to R n by 
considering the set S satisfying 
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where each a equals 0, -j, or -j, and no two of them ever equal at the same time. Use a computer to construct the set 
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thereby determining the value of m n for n = 2, 3, 4. Then develop an expression for m n . 


Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 

































11.15 

CHAOS 


In this section we use a map of the unit square in the xy-plane onto itself to 
describe the concept of a chaotic mapping. 


Geometry of Linear Operators on p 2 (Section 9.2) 

Eigenvalues and Eigenvectors 

Intuitive Understanding of Limits and Continuity 


Chaos 


The word chaos was first used in a mathematical sense in 1975 by Tien-Yien Li and James Yorke in a paper entitled “Period Three 
Implies Chaos.” The term is now used to describe the behavior of certain mathematical mappings and physical phenomena that at 
first glance seem to behave in a random or disorderly fashion but actually have an underlying element of order (examples include 
random-number generation, shuffling cards, cardiac arrhythmia, fluttering airplane wings, changes in the red spot of Jupiter, and 
deviations in the orbit of Pluto). In this section we discuss a particular chaotic mapping called Arnold's cat map , after the Russian 
mathematician Vladimir I. Arnold who first described it using a diagram of a cat. 


Arnold's Cat Map 


To describe Arnold's cat map, we need a few ideas about modular arithmetic. If x is a real number, then the notation x mod 1 
denotes the unique number in the interval [0, 1) that differs from x by an integer. For example, 

2.3mod 1 = 0.3, 0.9mod 1 = 0.9, - 3.7 mod 1 = 0.3, 2.0modl = 0 

Note that if x is a nonnegative number, then x mod 1 is simply the fractional part of x. If ( x , 7 ) is an ordered pair of real numbers, 
then the notation (*, y) mod 1 denotes (x mod 1, y mod 1). For example, 

(2.3, - 7.9) mod 1 = (0.3, 0.1) 

Observe that for every real number x, the point x mod 1 lies in the unit interval [0,1) and that for every ordered pair ( x , y). the 
point (x, yj) mod 1 lies in the unit square 

S= {(x, jy)|0 <x < 1, 0 < 1} 

Also observe that the upper boundary and the right-hand boundary of the square are not included in S. 


Arnold's cat map is the transformation jp p 2 _, p 2 defined by the formula 

F: ( x , .y) —» (x -\-y,x + 2 y) mod 1 


or, in matrix notation. 
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To understand the geometry of Arnold's cat map, it is helpful to write 1 in the factored form 
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which expresses Arnold's cat map as the composition of a shear in the x-direction with factor 1, followed by a shear in the 
j’-direction with factor 1. Because the computations are performed mod 1, r maps all points of pf 2 into the unit square S. 


( 1 ) 


We will illustrate the effect of Arnold's cat map on the unit square S, which is shaded in Figure 11.15.1a and contains a picture of a 
cat. It can be shown that it does not matter whether the mod 1 computations are carried out after each shear or at the very end. We 
will discuss both methods, first performing them at the end. The steps are as follows: 
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Figure 11.15.1 


Step 1. Shear in the x-direction with factor 1 (Figure 11.15.1ft): 

(x.y) —► (x+.y,.y) 

or in matrix notation 
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Step 2. Shear in the v-direction with factor 1 (Figure 11.15.1c): 

(x,y) -* (x, x+y) 

or, in matrix notation, 
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X 4 y 


Step 3. Reassembly into S (Figure 11.15.1J): 


(x,jy) —> (x, 7 ) modi 

The geometric effect of the mod 1 arithmetic is to break up the parallelogram in Figure 11.15.1c and 
reassemble the pieces of S as shown in Figure 11.15.Id. 

For computer implementation, it is more convenient to perform the mod 1 arithmetic at each step, rather than at the end. With this 
approach there is a reassembly at each step, but the net effect is the same. The steps are as follows: 


Step 1. Shear in the x-direction with factor 1, followed by a reassembly into S (Figure 11.15.2/?): 

0,.y) —* (x + y,y) mod 1 


Step 2. Shear in the y-direction with factor 1, followed by a reassembly into S (Figure 11.15.2c): 


(x, y) -> (x, x 4 7 ) mod 1 
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Figure 11.15.2 


Repeated Mappings 

Chaotic mappings such as Arnold's cat map usually arise in physical models in which an operation is performed repeatedly. For 
example, cards are mixed by repeated shuffles, paint is mixed by repeated stirs, water in a tidal basin is mixed by repeated tidal 
changes, and so forth. Thus, we are interested in examining the effect on S of repeated applications (or iterations) of Arnold's cat 
map. Figure 11.15.3, which was generated on a computer, shows the effect of 25 iterations of Arnold's cat map on the cat in the 
unit square S. Two interesting phenomena occur: 


The cat returns to its original form at the 25th iteration. 


At some of the intermediate iterations, the cat is decomposed into streaks that seem to have a specific direction. 


Much of the remainder of this section is devoted to explaining these phenomena. 
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Figure 11.15.3 


Periodic Points 


Our first goal is to explain why the cat in Figure 11.15.3 returns to its original con. guration at the 25th iteration. For this purpose it 
will be helpful to think of a picture in the xy-plane as an assignment of colors to the points in the plane. For pictures generated on 
a computer screen or other digital device, hardware limitations require that a picture be broken up into discrete squares, called 
pixels. For example, in the computer-generated pictures in Figure 11.15.3 the unit square S is divided into a grid with 101 pixels on 
a side for a total of 10,201 pixels, each of which is black or white (Figure 11.15.4). An assignment of colors to pixels to create a 
picture is called a pixel map. 



hnlaqied view of cat's tikuv 
shutting individual pixels 



Figure 11.15.4 

As shown in Figure 11.15.5, each pixel in S can be assigned a unique pair of coordinates of the form (w*/101>m/ 101) that 
identifies its lower left-hand comer, where m and n are integers in the range 0, 1,2, ..., 100. We call these points pixel points 




















because each such point identifies a unique pixel. Instead of restricting the discussion to the case where S is subdivided into an 
array with 101 pixels on a side, let us consider the more general case where there are p pixels per side. Thus, each pixel map in S 

"i 

consists of p pixels uniformly spaced 1 ip units apart in both the x- and the y-directions. The pixel points in S have coordinates of 
the form (m / p, n / p) where m and n are integers ranging from 0 to p — 1 . 



V _j_ 1 _ t + . Jtf_ + i MJ 

Lm i 4 ii mil lli i mu mi 


Figure 11.15.5 


Under Arnold's cat map each pixel point of S is transformed into another pixel point of S. To see why this is so, observe that the 
image of the pixel point {m i p, n ! p) under r is given in matrix form by 
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The ordered pair (( m 4 n) i p, (m 4 - 2n) / p) mod 1 is of the form (m ! ! p,n ! ! p), where yn and n ! lie in the range 0, 1,2, ..., 
p — 1. Specifically, m ! and n ! are the remainders when m \- n and m -\- 2n are divided by p, respectively. Consequently, each point 
in S of the form (m / p, n i p) is mapped onto another point of the same form. 


Because Arnold's cat map transforms every pixel point of S into another pixel point of S, and because there are only p different 
pixel points in S, it follows that any given pixel point must return to its original position after at most p iterations of Arnold's cat 
map. 


Using Formula 2 


If p = 76, then 2 becomes 
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(verify). Because the point returns to its initial position on the ninth application of Arnold's cat map (but no sooner), the point is 
said to have period 9, and the set of nine distinct iterates of the point is called a 9-cycle. Figure 11.15.6 shows this 9-cycle with the 
initial point labeled 0 and its successive iterates labeled accordingly. 















































Figure 11.15.6 
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In general, a point that returns to its initial position after n applications of Arnold's cat map, but does not return with fewer than n 
applications, is said to have period n, and its set of n distinct iterates is called an n-cycle. Arnold's cat map maps (0, 0) into (0, 0), 
so this point has period 1. Points with period 1 are also called fixed points. We leave it as an exercise (Exercise 11) to show that 
(0, 0) is the only fixed point of Arnold's cat map. 

Period versus Pixel Width 

If Pj and Pj are points with periods q\ and # 2 , respectively, then p^ returns to its initial position in q\ iterations (but no sooner), 
and P 2 returns to its initial position in #2 iterations (but no sooner); thus, both points return to their initial positions in any number 

“1 

of iterations that is a multiple of both q 1 and # 2 - In general, for a pixel map with p pixel points of the form {m / p,n i p), we let 
n(^) denote the least common multiple of the periods of all the pixel points in the map [that is, Yl(p) is the smallest integer that is 
divisible by all of the periods]. It follows that the pixel map will return to its initial configuration in H(p) iterations of Arnold's cat 
map (but no sooner). For this reason, we call II(^) the period of the pixel map. In Exercise 4 we ask the reader to show that if 
p = 101, then all pixel points have period 1, 5, or 25, so n(101) = 25- This explains why the cat in Figure 11.15.3 returned to its 
initial configuration in 25 iterations. 

Figure 11.15.7 shows how the period of a pixel map varies with p. Although the general tendency is for the period to increase as p 
increases, there is a surprising amount of irregularity in the graph. Indeed, there is no simple function that specifies this 
relationship (see Exercise 1). 



Figure 11.15.7 

Although a pixel map with p pixels on a side does not return to its initial configuration until U(p) iterations have occurred, various 
unexpected things can occur at intermediate iterations. For example, Figure 11.15.8 shows a pixel map with p = 250 of the famous 
Hungarian-American mathematician John von Neumann. It can be shown that n(250) = 750; hence, the pixel map will return to 
its initial configuration after 750 iterations of Arnold's cat map (but no sooner). However, after 375 iterations the pixel map is 
turned upside down, and after another 375 iterations (for a total of 750) the pixel map is returned to its initial configuration. 




























































Moreover, there are so many pixel points with periods that divide 750 that multiple ghostlike images of the original likeness occur 
at intermediate iterations; at 195 iterations numerous miniatures of the original likeness occur in diagonal rows. 


250 pixels 
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Figure 11.15.8 


The Tiled Plane 


Our next objective is to explain the cause of the linear streaks that occur in Figure 11.15.3. For this purpose it will be helpful to 
view Arnold's cat map another way. As defined, Arnold's cat map is not a linear transformation because of the mod 1 arithmetic. 
However, there is an alternative way of defining Arnold's cat map that avoids the mod 1 arithmetic and results in a linear 
transformation. For this purpose, imagine that the unit square S with its picture of the cat is a “tile,” and suppose that the entire 
plane is covered with such tiles, as in Figure 11.15.9. We say that the xy-plane has been tiled with the unit square. If we apply the 
matrix transformation in 1 to the entire tiled plane without performing the mod 1 arithmetic, then it can be shown that the portion 
of the image within S will be identical to the image that we obtained using the mod 1 arithmetic (Figure 11.15.9). In short, the 
tiling results in the same pixel map in S as the mod 1 arithmetic, but in the tiled case Arnold's cat map is a linear transformation. 



Figure 11.15.9 


It is important to understand, however, that tiling and mod 1 arithmetic produce periodicity in different ways. If a pixel map in S 
has period n, then in the case of mod 1 arithmetic, each point returns to its original position at the end of n iterations. In the case of 
tiling, points need not return to their original positions; rather, each point is replaced by a point of the same color at the end of n 





































iterations. 


Properties of Arnold's Cat Map 

To understand the cause of the streaks in Figure 11.15.3, think of Arnold's cat map as a linear transformation on the tiled plane. 
Observe that the matrix 


that defines Arnold's cat map is symmetric and has a determinant of 1. The fact that the determinant is 1 means that multiplication 
by this matrix preserves areas; that is, the area of any figure in the plane and the area of its image are the same. This is also true for 
figures in S in the case of mod 1 arithmetic, since the effect of the mod 1 arithmetic is to cut up the figure and reassemble the 
pieces without any overlap, as shown in Figure 11.15. Id. Thus, in Figure 11.15.3 the area of the cat (whatever it is) is the same as 
the total area of the blotches in each iteration. 

The fact that the matrix is symmetric means that its eigenvalues are real and the corresponding eigenvectors are perpendicular. We 
leave it for the reader to show that the eigenvalues and corresponding eigenvectors of C are 


A t = 3+ 2 ^ =2.6180-, A 2 = 3 =0.3819-, 
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For each application of Arnold's cat map, the eigenvalue Aj causes a stretching in the direction of the eigenvector by a factor of 
2.6180 ..., and the eigenvalue A 2 causes a compression in the direction of the eigenvector v 2 by a factor of 0.3819 .... Figure 
11.15.10 shows a square centered at the origin whose sides are parallel to the two eigenvector directions. Under the above 
mapping, this square is deformed into the rectangle whose sides are also parallel to the two eigenvector directions. The area of the 
square and rectangle are the same. 



Figure 11.15.10 

To explain the cause of the streaks in Figure 11 . 15 . 3 , consider S to be part of the tiled plane, and let p be a point of S with period n. 
Because we are considering tiling, there is a point q in the plane with the same color as p that on successive iterations moves 
toward the position initially occupied by p, reaching that position on the nth iteration. This point is q = (A~^) p =A~ n p, since 

A\ = A”(A~” p)=p 

Thus, with successive iterations, points of S flow away from their initial positions, while at the same time other points in the plane 
(with corresponding colors) flow toward those initial positions, completing their trip on the final iteration of the cycle. Figure 

11.15.11 illustrates this in the case where « = 4, q = ^ — y, y j, and p = , j j. Note that p mod 1 = q mod 1 = , j j, so 

both points occupy the same positions on their respective tiles. The outgoing point moves in the general direction of the 
eigenvector vp as indicated by the arrows in Figure 11 . 15 . 11 , and the incoming point moves in the general direction of eigenvector 
v 2 - It is the “flow lines” in the general directions of the eigenvectors that form the streaks in Figure 11 . 15 . 3 . 
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Figure 11.15.11 


Nonperiodic Points 

Thus far we have considered the effect of Arnold's cat map on pixel points of the form (m ! p, n i p) for an arbitrary positive 
integer p. We know that all such points are periodic. We now consider the effect of Arnold's cat map on an arbitrary point (a, b ) in 
S. We classify such points as rational if the coordinates a and b are both rational numbers, and irrational if at least one of the 
coordinates is irrational. Every rational point is periodic, since it is a pixel point for a suitable choice of p. For example, the 
rational point (^ / ^ can written as (r\S 2 / s\S 2 > so ' s a pixel point with p = It can be shown 

(Exercise 13) that the converse is also true: Every periodic point must be a rational point. 

It follows from the preceding discussion that the irrational points in S are nonperiodic, so that successive iterates of an irrational 
point (xq, yo) in S must all be distinct points in S. Figure 11.15.12, which was computer-generated, shows an irrational point and 
selected iterates up to 100,000. For the particular irrational point that we selected, the iterates do not seem to cluster in any 
particular region of 5; rather, they appear to be spread throughout S, becoming denser with successive iterations. 




Figure 11.15.12 

The behavior of the iterates in Figure 11.15.12 is sufficiently important that there is some terminology associated with it. We say 
that a set D of points in S is dense in S if every circle centered at any point of S encloses points of D, no matter how small the 
radius of the circle is taken (Figure 11.15.13). It can be shown that the rational points are dense in S and the iterates of most (but 



































not all) of the irrational points are dense in S. 
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Definition of Chaos 

We know that under Arnold's cat map, the rational points of S are periodic and dense in S and that some but not all of the irrational 
points have iterates that are dense in S. These are the basic ingredients of chaos. There are several definitions of chaos in current 
use, but the following one, which is an outgrowth of a definition introduced by Robert L. Devaney in 1986 in his book An 
Introduction to Chaotic Dynamical Systems (Benjamin/Cummings Publishing Company), is most closely related to our work. 


DEFINITION 


A mapping T of S onto itself is said to be chaotic if: 

(i) S contains a dense set of periodic points of the mapping T. 

(ii) There is a point in S whose iterates under T are dense in S. 


Thus Arnold's cat map satisfies the definition of a chaotic mapping. What is noteworthy about this definition is that a chaotic 
mapping exhibits an element of order and an element of disorder—the periodic points move regularly in cycles, but the points with 
dense iterates move irregularly, often obscuring the regularity of the periodic points. This fusion of order and disorder 
characterizes chaotic mappings. 

Dynamical Systems 

Chaotic mappings arise in the study of dynamical systems. Informally stated, a dynamical system can be viewed as a system that 
has a specific state or configuration at each point of time but that changes its state with time. Chemical systems, ecological 
systems, electrical systems, biological systems, economic systems, and so forth can be looked at in this way. In a discrete-time 
dynamical system, the state changes at discrete points of time rather than at each instant. In a discrete-time chaotic dynamical 
system, each state results from a chaotic mapping of the preceding state. For example, if one imagines that Arnold's cat map is 
applied at discrete points of time, then the pixel maps in Figure 11.15.3 can be viewed as the evolution of a discrete-time chaotic 
dynamical system from some initial set of states (each point of the cat is a single initial state) to successive sets of states. 

One of the fundamental problems in the study of dynamical systems is to predict future states of the system from a known initial 
state. In practice, however, the exact initial state is rarely known because of errors in the devices used to measure the initial state. It 
was believed at one time that if the measuring devices were sufficiently accurate and the computers used to perform the iteration 
were sufficiently powerful, then one could predict the future states of the system to any degree of accuracy. But the discovery of 
chaotic systems shattered this belief because it was found that for such systems the slightest error in measuring the initial state or in 
the computation of the iterates becomes magnified exponentially, thereby preventing an accurate prediction of future states. Let us 
demonstrate this sensitivity to initial conditions with Arnold's cat map. 











Suppose that p g is a point in the xy-plane whose exact coordinates are (0.77837, 0.70904). A measurement error of 0.00001 is 
made in the y-coordinate, such that the point is thought to be located at (0.77837, 0.70905), which we denote by Qq. Both and 
Qq are pixel points with p = 100,000 (why?), and thus, since n(l00,000) = 75,000> both return to their initial positions after 
75,000 iterations. In Figure 11.15.14 we show the first 50 iterates of P 0 under Arnold's cat map as crosses and the first 50 iterates 
of Qq as circles. Although p Q and Qq are close enough that their symbols overlap initially, only their first eight iterates have 
overlapping symbols; from the ninth iteration on their iterates follow divergent paths. 



It is possible to quantify the growth of the error from the eigenvalues and eigenvectors of Arnold's cat map. For this purpose we 
shall think of Arnold's cat map as a linear transformation on the tiled plane. Recall from Figure 11.15.10 and the related discussion 
that the projected distance between two points in S in the direction of the eigenvector vq increases by a factor of 2.6180...( = Ai) 
with each iteration (Figure 11.15.15). After nine iterations this projected distance increases by a factor of (2.61 SO...) 5 = 5777.99..., 

and with an initial error of roughly 1/100,000 in the direction of vp this distance is 0.05777 ..., or about yy the width of the unit 
square S. After 12 iterations this small initial error grows to (2.6180...) 12 / 100,000 = 1.0368..., which is greater than the width of 

S. Thus, we lose complete track of the true iterates within S after 12 iterations because of the exponential growth of the initial 
error. 



Figure 11.15.15 

Although sensitivity to initial conditions limits the ability to predict the future evolution of dynamical systems, new techniques are 
presently being investigated to describe this future evolution in alternative ways. 


Exercise Set 11.15 


Click here for Just Ask! 




In a journal article [F. J. Dyson and H. Falk, “Period of a Discrete Cat Mapping,” The American Mathematical Monthly, 99 
!• (August-September 1992), pp. 603-614] the following results concerning the nature of the function Yl(p) were established: 


(i) FI(_p) = 3 p if and only if p = 2 • 5 k for k = 1, 2,.... 

(ii) ElO) = 2p if and only if ;? = 5 * for £ = 1, 2,... or p = 6 - 5* for k = 0, 1, 2,...- 

(iii) FI(_p) < 12p / 7 for all other choices of p. 

Find 11(250), 11(25), 11(125), 11(30), 11(10), 11(50), 11(3750), 11(6), and n(5). 

Find all the »-cycles that are subsets of the 36 points in S of the form (w* / 6, « / 6) with m and n in the range 0, 1, 2, 3, 4, 5. 

2. Then find n(6)- 


3. (Fibonacci Shift-Register Random-Number Generator) A well-known method of generating a sequence of 
“pseudorandom” integers xq, x\, ?:?, xj, ... in the interval from 0 to p — \ is based on the following algorithm: 

(i) Pick any two integers xq and jq from the range 0, 1,2, ..., p — 1. 

(ii) Set x H+1 = (x M + x H _!) mod p for n = 1, 2,.... 

Here x mod p denotes the number in the interval from 0 to p — 1 that differs from x by a multiple of p. For example, 

35 mod 9 = 8 (because 8 = 35 - 3 ■ 9); 36 mod 9 = 0 (because 0 = 36 - 4 - 9); and _ 3 mod 9 = 6 (because 6 = - 3 + 1 - 9). 

(a) Generate the sequence of pseudorandom numbers that results from the choices p — 15, = 3> and x\=l until the 

sequence starts repeating. 


(b) Show that the following formula is equivalent to step (ii) of the algorithm: 


^H + l 


'1 

2 ' 

—1 

7 l " h +2 


_1 

2 _ 



mod_p 


forw = 1, 2, 3,... 


(c) Use the formula in part (b) to generate the sequence of vectors for the choices p = 21, xq = 5, and = 5 until the 
sequence starts repeating. 


If we take p — 1 and pick *o and x\ from the interval [0, 1), then the above random-number generator produces 
pseudorandom numbers in the interval [0, 1). The resulting-scheme is precisely Arnold's cat map. Furthermore, if we eliminate 
the modular arithmetic in the algorithm and take = 1, then the resulting sequence of integers is the famous Fibonacci 

sequence, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ..., in which each number after the first two is the sum of the preceding two 
numbers. 


1 1 
1 2 


4. 


For C = 


, it can be verified that 










c 


25 


7,778,742,049 12,586,269,025 
12,586,269,025 20,365,011,074 


It can also be verified that 12,586,269,025 is divisible by 101 and that when 7,778,742,049 and 20,365,011,074 are divided by 
101, the remainder is 1. 


(a) Show that every point in S of the form (m/101>«/101) returns to its starting position after 25 iterations under Arnold's 
cat map. 


(b) Show that every point in S of the form Gw/101>«/101) ^ as P er iod 1, 5, or 25. 


Show that the point ^ ^ ; o j has period greater than 5 by iterating it five times. 


(d) Show that n( 101 ) = 25- 


Show that for the mapping T: S —» 5 defined by T(x, y ) 
does this show that the mapping is not chaotic? 



mod 1, every point in S is a periodic point. Why 


6 . 


An Anosov automorphism on p} is a mapping from the unit square S onto S of the form 

mod 1 


a b 
c d 


in which (i) a, b, c, and d are integers, (ii) the determinant of the matrix is ± 1, an d (iii) the eigenvalues of the matrix do not 
have magnitude 1. It can be shown that all Anosov automorphisms are chaotic mappings. 


(a) Show that Arnold's cat map is an Anosov automorphism. 


(b) Which of the following are the matrices of an Anosov automorphism? 


'0 f 


'3 2 ' 


'1 o' 


'5 

1 


'6 

2' 

_1 0_ 


_1 1_ 


_0 1_ 


2 

3_ 


_5 

2_ 


(c) Show that the following mapping of S onto S is not an Anosov automorphism. 


x 

What is the geometric effect of this transformation on S? Use your observation to show that the 
mapping is not a chaotic mapping by showing that all points in S are periodic points. 


0 1 

“x “ 

-1 0 _ 

y 


mod 1 


Show that Arnold's cat map is one-to-one over the unit square S and that its range is S. 

7. 


Show that the inverse of Arnold's cat map is given by 
r -1 (x, 7 ) = ( 2 x-y, - X I 7 ) mod 1 


9 . 


Show that the unit square S can be partitioned into four triangular regions on each of which Arnold's cat map is a 


transformation of the form 


~x " 


'i r 

'x" 


"a" 

j 


.i 2 _ 

y 

+ 

b_ 


where a and b need not be the same for each region. 

Hint Find the regions in S that map onto the four shaded regions of the parallelogram in Figure 11.15.1 d. 


10 . 


x n 


'1 

f 


'*0' 

Jh 


_1 

2_ 


.TO 


11 . 


If (*0’ Jo) i s a point in S and (x H , y n ) is its nth iterate under Arnold's cat map, show that 

mod 1 

This result implies that the modular arithmetic need only be performed once rather than after each iteration. 

Show that (0, 0) is the only fixed point of Arnold's cat map by showing that the only solution of the equation 

mod 1 


■*o' 


'1 

f 

'*0' 

JO 


_1 

2_ 

JO 


with 0 < * 0 < 1 and 0 <y 0 < 1 is x 0 = y 0 = 0- 

Hint For appropriate nonnegative integers, r and s, the preceding equation can be written as 


■*o' 


'i r 

■*o' 



JO 


.i 2 _ 

JO 


s 


12 . 


Find all 2-cycles of Arnold's cat map by finding all solutions of the equation 

i ri l - pP'oi 

/or |.i 2J W 

with 0 < x 0 < 1 and 0 <y 0 < 1- 

Hint For appropriate nonnegative integers, r and s, the preceding equation can be written as 


'*0' 


'2 3' 

'*0' 



JO 


i 

U.j 

'-•-I 

I_ 

J0 


S 


13. 


Show that every periodic point of Arnold's cat map must be a rational point by showing that for all solutions of the equation 


~xa~ 


'1 

r 

n 


_J0_ 


_1 

2_ 


_JG_ 


mod 1 


the numbers xq and yg are quotients of integers. 


Section 11.15 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


















































Tl. 


The methods of Exercise 4 show that for the cat map, n(;>) is the smallest integer satisfying the equation 


1 1 
1 2 


n&> 


mod;? = 


1 0 
0 1 


This suggests that one way to determine n(;>) is to compute 


1 1 
1 2 


mod;? 


starting with « = 1 and stopping when this produces the identity matrix. Use this idea to compute Yl(p) for p = 2, 3,10- 
Compare your results to the formulas given in Exercise Tl, if they apply. What can you conjecture about 


1 1 
1 2 


fno?) 


mod;? 


T2. 


when no?) is even? 

The eigenvalues and eigenvectors for the cat map matrix 

C = 


1 1 
1 2 


are 




= 1±JL ,y 

2 ’ 2 2 



1 


1 

VI = 

14 i/5 

v 2 = 



2 


2 


Using these eigenvalues and eigenvectors, we can define 



\2±iH 0 


1 1 

D = 

2 

0 i=JL 

and P = 

lH-l/5 l-i/5 

2 2 


2 




and write C = PDP~^'> hence, £”=/?£} Use a computer to show that 

C" = 


(») (») 
C 11 c 12 

(») (») 
c 2\ c 22 


where 


L-n — 


~(«) _ 
u 22 — 



j 1-^5 \ 


) 

l 2 ) l 2/5 J 

1 2 J 


/3+l/5r 


2/5 ) 

l 2 J l 2/5 J 

l 2 J 


and 


e «_4?--ute±£ I'.fMf 




\ 


2 I \ 2 


How can you use these results and your conclusions in Exercise Tl to simplify the method for computing n(/>)? 


Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 



11.16 

CRYPTOGRAPHY 


In this section we present a method of encoding and decoding messages. We 
also examine modular arithmetic and show how Gaussian elimination can 
sometimes be used to break an opponent's code. 


Matrices 

Gaussian Elimination 
Matrix Operations 
Linear Independence 

Linear Transformations (Sections 8.1 and 8.2) 


Ciphers 

The study of encoding and decoding secret messages is called cryptography . Although secret codes date to the earliest days of 
written communication, there has been a recent surge of interest in the subject because of the need to maintain the privacy of 
information transmitted over public lines of communication. In the language of cryptography, codes are called ciphers, uncoded 
messages are called plaintext, and coded messages are called ciphertext. The process of converting from plaintext to ciphertext is 
called enciphering, and the reverse process of converting from ciphertext to plaintext is called deciphering. 


The simplest ciphers, called substitution ciphers, are those that replace each letter of the alphabet by a different letter. Lor 
example, in the substitution cipher 

Plain ABCDEFGHI JKLMNOPQRS T U V W 

Cipher D E F G H I JKLMNOPQRS T U V W X Y Z 

the plaintext letter A is replaced by D, the plaintext letter B by E, and so forth. With this cipher the plaintext message 

ROME WAS NOT BUILT IN A DAY 


becomes 


URPH ZDV QRW EX LOW LQ D GDB 


X 

A 


Hill Ciphers 

A disadvantage of substitution ciphers is that they preserve the frequencies of individual letters, making it relatively easy to break 
the code by statistical methods. One way to overcome this problem is to divide the plaintext into groups of letters and encipher the 
plaintext group by group, rather than one letter at a time. A system of cryptography in which the plaintext is divided into sets of n 
letters, each of which is replaced by a set of n cipher letters, is called a polygraphic system. In this section we will study a class of 
polygraphic systems based on matrix transformations. (The ciphers that we will discuss are called Hill ciphers after Lester S. Hill, 
who introduced them in two papers: “Cryptography in an Algebraic Alphabet,” American Mathematical Monthly, 36 (June- July 
1929), pp. 306-312; and “Concerning Certain Linear Transformation Apparatus of Cryptography,” American Mathematical 
Monthly, 38 (March 1931), pp. 135-154.) 

In the discussion to follow, we assume that each plaintext and ciphertext letter except Z is assigned the numerical value that 
specifies its position in the standard alphabet (Table 1). Lor reasons that will become clear later, Z is assigned a value of zero. 


Table 1 






A 

B 

C 

D 

E 

F 

G 

H 

I 

J 

K 

L 

M 

N 

0 

P 

Q 

R 

S 

T 

U 

V 

W 

X 

Y 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

23 

24 

25 


In the simplest Hill ciphers, successive pairs of plaintext are transformed into ciphertext by the following procedure: 


Step 1. Choose a 2 x 2 matrix with integer entries 


AH a 12 
«21 a 22 


to perform the encoding. Certain additional conditions on A will be imposed later. 


Step 2. Group successive plaintext letters into pairs, adding an arbitrary “dummy” letter to fill out the last pair if the plaintext 
has an odd number of letters, and replace each plaintext letter by its numerical value. 


Step 3. Successively convert each plaintext pair p\p 2 into a column vector 

>r 

V= [P2_ 

and form the product Ap. We will call p a plaintext vector and Ay the corresponding ciphertext 
vector. 

Step 4. Convert each ciphertext vector into its alphabetic equivalent. 


Hill Cipher of a Message 


Use the matrix 

'1 2 ' 

_° 3_ 

to obtain the Hill cipher for the plaintext message 

l AM HIDING 


Solution 


If we group the plaintext into pairs and add the dummy letter G to fill out the last pair, we obtain 


IA MR ID IN GG 

or, equivalently, from Table 1, 

9 1 138 94 9 14 77 

To encipher the pair /A, we form the matrix product 


'1 2' 

'9' 


'll' 

- 1 

m 

o 

_i 

_1_ 


_ 3 _ 


which, from Table 1, yields the ciphertext KC- 


To encipher the pair MR, we form the product 



















However, there is a problem here, because the number 29 has no alphabet equivalent (Table 1). To resolve this problem, we make 
the following agreement: 


Whenever an integer greater than 25 occurs, it will be replaced by the remainder that results when this integer is 
divided by 26. 

Because the remainder after division by 26 is one of the integers 0, 1,2, ..., 25, this procedure will always yield an integer with an 
alphabet equivalent. 

Thus, in 1 we replace 29 by 3, which is the remainder after dividing 29 by 26. It now follows from Table 1 that the ciphertext for 
the pair MR is CX 

The computations for the remaining ciphertext vectors are 


'1 2' 

rs' 


'17' 

i 

m 

O 

_i 

|_4_ 


_12_ 

1 2] 

' 9 ' 


'37" 

m ' 

o 

14_ 


_42_ 

'1 2' 

P?' 


'21' 

O 

_ i 

7 


21 


or 


11 

16 


These correspond to the ciphertext pairs QL, ?fP, and JJJJ, respectively. In summary, the entire ciphertext message is 

KC CX QL KP UU 
which would usually be transmitted as a single string without spaces: 

KCCXQLKPUU 


Because the plaintext was grouped in pairs and enciphered by a 2 x 2 matrix, the Hill cipher in Example 1 is referred to as a Hill 
2-cipher. It is obviously also possible to group the plaintext in triples and encipher by a 3 x 3 matrix with integer entries; this is 
called a Hill 3-cipher. In general, for a Hill n-cipher, plaintext is grouped into sets of n letters and enciphered by an n x n matrix 
with integer entries. 

Modular Arithmetic 

In Example 1, integers greater than 25 were replaced by their remainders after division by 26. This technique of working with 
remainders is at the core of a body of mathematics called modular arithmetic. Because of its importance in cryptography, we will 
digress for a moment to touch on some of the main ideas in this area. 

In modular arithmetic we are given a positive integer m, called the modulus , and any two integers whose difference is an integer 
multiple of the modulus are regarded as “equal” or “equivalent” with respect to the modulus. More precisely, we make the 
following definition. 


DEFINITION 


If m is a positive integer and a and b are any integers, then we say that a is equivalent to b modulo m, written 

a = b (raodm) 

if a — b is an integer multiple of m. 


































Various Equivalences 


7 = 2 

(mod 5) 

19 = 3 

(mod 2 ) 

- 1 = 25 

(mod 26) 

12 = 0 

(mod 4) 


For any modulus m it can be proved that every integer a is equivalent, modulo m, to exactly one of the integers 

0 , 1 , 1 

We call this integer the residue of a modulo m, and we write 

Z m = {0, 1, 

to denote the set of residues modulo m. 

If a is a nonnegative integer, then its residue modulo m is simply the remainder that results when a is divided by m. For an 
arbitrary integer a, the residue can be found using the following theorem. 


THEOREM 11.16.1 


For any integer a and modulus m, let 





R = 

= remainder 

ofM 




m 

Then the residue r of a modulo m 

is given by 




(* 

if a > 0 



r = l m = 

R if a < 0 

and R ^ 0 


1 ° 

if a < 0 

and R = 0 


EXAMPLE 3 Residues mod 26 


Find the residue modulo 26 of (a) 87, (b) -38, and (c) -26. 


Solution (a) 

Dividing |87| = 87 by 26 yields a remainder of R = 9 , so r = 9 . Thus, 

87 = 9 (mod 26) 


Solution (b) 

Dividing | — 381 = 38 by 26 yields a remainder of R = 12 , so r = 26 — 12 = 14- Thus, 

-38 = 14 (mod 26) 


Solution (c) 

Dividing | — 261 = 26 by 26 yields a remainder of £ = 0 - Thus, 






-26 = 0 (mod 26) 


♦ 


In ordinary arithmetic every nonzero number a has a reciprocal or multiplicative inverse, denoted by a such that 

= 1 

In modular arithmetic we have the following corresponding concept: 


DEFINITION 


If a is a number in z m , then a number fl _1 in z m is called a reciprocal or multiplicative inverse of a modulo m if 
= 1 (modm). 


It can be proved that if a and m have no common prime factors, then a has a unique reciprocal modulo m; conversely, if a and m 
have a common prime factor, then a has no reciprocal modulo m. 


XA Reciprocal of 3 mod 26 


The number 3 has a reciprocal modulo 26 because 3 and 26 have no common prime factors. This reciprocal can be obtained by 
finding the number x in z^ that satisfies the modular equation 

2x = 1 (mod 26) 


Although there are general methods for solving such modular equations, it would take us too far afield to study them. However, 
because 26 is relatively small, this equation can be solved by trying the possible solutions, 0 to 25, one at a time. With this 
approach we find that x = 9 is the solution, because 

3 - 9 = 27 = 1 (mod 26) 


Thus, 


3 -1 = 9 (mod 26) 


♦ 


EXAMPLE 5 A Number with No Reciprocal mod 26 


The number 4 has no reciprocal modulo 26, because 4 and 26 have 2 as a common prime factor (see Exercise 8). 


For future reference, we provide the following table of reciprocals modulo 26: 


Table 2 

Reciprocals Modulo 26 


a 

1 

3 

5 

7 

9 

11 

15 

17 

19 

21 

23 

25 


1 

9 

21 

15 

3 

19 

7 

23 

11 

5 

17 

25 


Deciphering 













Every useful cipher must have a procedure for decipherment. In the case of a Hill cipher, decipherment uses the inverse (mod 26) 
of the enciphering matrix. To be precise, if m is a positive integer, then a square matrix A with entries in z,,, is said to be invertible 
modulo m if there is a matrix B with entries in 7 such that 

"Tfl 

AB = BA = l (modtfs) 


Suppose now that 

A _\ a n an 

' [<321 a 22 

is invertible modulo 26 and this matrix is used in a Hill 2-cipher. If 

>f 


is a plaintext vector, then 


c=j4p (mod 26) 


is the corresponding ciphertext vector and 

P = j4 -1 c (mod 26) 


Thus, each plaintext vector can be recovered from the corresponding ciphertext vector by multiplying it on the left by 
j4 -1 (mod 26). 


In cryptography it is important to know which matrices are invertible modulo 26 and how to obtain their inverses. We now 
investigate these questions. 

In ordinary arithmetic, a square matrix A is invertible if and only if det(2l) 0> or, equivalently, if and only if det(A) has a 
reciprocal. The following theorem is the analog of this result in modular arithmetic. 


THEOREM 11.16.2 


A square matrix A with entries in Z m is invertible modulo m if and only if the residue of det(A) modulo m has a reciprocal 
modulo m. 


Because the residue of det(2l) modulo m will have a reciprocal modulo m if and only if this residue and m have no common prime 
factors, we have the following corollary. 


COROLLARY 11.16.3 


A square matrix A with entries in Z m is invertible modulo m if and only ifm and the residue ofdet(A) modulo m have no 
common prime factors. 


Because the only prime factors of m = 26 are 2 and 13, we have the following corollary, which is useful in cryptography. 


COROLLARY 11.16.4 


A square matrix A with entries in Z-^ i s invertible modulo 26 if and only if the residue of det(A) modulo 26 is not divisible by 2 
or 13. 









We leave it for the reader to verify that if 


A = 


a b 
c d 


has entries in anc * the residue of det(j4) = ad — be modulo 26 is not divisible by 2 or 13, then the inverse of A (mod 26) is 
given by 


A -1 = (ad-be) -1 


d -b 

— c a 


(mod 26) 


where (ad — be) 1 is the reciprocal of the residue of ad — be (mod 26). 


(2) 


Inverse of a Matrix mod 26 


Find the inverse of 


modulo 26. 


Solution 


A = 


5 6 
2 3 


so from Table 2, 


Thus, from 2, 


As a check. 


det (A)=ad-bc = 5-3 — 6-2 = 3 


(ad-be) -1 = 3 _1 = 9 (mod26) 


1 — Q 

'3 -6' 


1 

[■■O 

---j 

1 

U-I 

_i 


1 

1_ 

— y 

1 

[O 

U-i 


-13 45 


8 19 


AA~ X = 


'5 6' 

'1 24' 


'53 234' 


l 

O 

-1 

r-"i 

_i 

1- 

CO 

V-O 

1_ 


26 105 


1 

o 

1_ 


(mod 26) 


(mod 26) 


Similarly, A -1 A = L 


EXAMf Decoding a Hill 2-Cipher 

Decode the following Hill 2-cipher, which was enciphered by the matrix in Example 6: 

GTNKGKDUSK 

Solution 

From Table 1 the numerical equivalent of this ciphertext is 

720 14 11 7 11 421 19 11 

To obtain the plaintext pairs, we multiply each ciphertext vector by the inverse of A (obtained in Example 6): 


























'1 

_8 

24' 

19_ 

1-1 

1 _ 1 

= 

'487' 

_436_ 

= 

'19' 

_20_ 

(mod 26) 

'1 

_8 

24' 

19_ 

'14' 

_11_ 

= 

CO i—' 

r- ■"■■■.] 

= 

'18' 
_ 9 _ 

(mod 26) 

'1 

_8 

24' 

19 _ 

' 7 ' 
_11_ 

= 

'271' 

265 

= 

'll' 
_ 5 _ 

(mod 26) 

'1 

_8 

24' 

19_ 

' 4 ' 
.21. 

= 

'508' 
431 _ 

= 

'14' 
_ 15_ 

(mod 26) 

'1 

3 

24' 

19 

'19' 

11 

= 

'283' 

361 

= 

'23' 

23 

(mod 26) 


From Table 1, the alphabet equivalents of these vectors are 

ST FI KB NO WW 

which yields the message 

STRIKE NOW 


Breaking a Hill Cipher 

Because the purpose of enciphering messages and information is to prevent “opponents” from learning their contents, 
cryptographers are concerned with the security of their ciphers—that is, how readily they can be broken (deciphered by their 
opponents). We will conclude this section by discussing one technique for breaking Hill ciphers. 

Suppose that you are able to obtain some corresponding plaintext and ciphertext from an opponent's message. For example, on 
examining some intercepted ciphertext, you may be able to deduce that the message is a letter that begins DEAR SIR. We will show 
that with a small amount of such data, it may be possible to determine the deciphering matrix of a Hill code and consequently 
obtain access to the rest of the message. 

It is a basic result in linear algebra that a linear transformation is completely determined by its values at a basis. This principle 
suggests that if we have a Hill n-cipher, and if 

P 1 -P 2 . —. 1>H 

are linearly independent plaintext vectors whose corresponding ciphertext vectors 

Api,Ap2,...,A[> n 

are known, then there is enough information available to determine the matrix A and hence A -1 (mod™). 

The following theorem, whose proof is discussed in the exercises, provides a way to do this. 


THEOREM 11.16.5 


Determining the Deciphering Matrix 

Let pi, p 2 > ..., p v, be linearly independent plaintext vectors, and let c\, 03 , •••, c H be the corresponding ciphertext vectors in a 
Hill n-cipher. If 



P = 


T 

1>2 


is the n-xn matrix with row vectors r, p t and if 















































i 

n 

^ h -3 

_i 



c = 

J 

c 2 




C T 

r H 


is the n-xn matrix with row vectors c ^, C J, . 

t T, then the sequence of elementary row operations 

that reduces C to I transforms P to (^ -1 ) T ■ 




This theorem tells us that to find the transpose of the deciphering matrix we must find a sequence of row operations that 
reduces C to I and then perform this same sequence of operations on P. The following example illustrates a simple algorithm for 
doing this. 


Using Theorem 11.16.5 


The following Hill 2-cipher is intercepted: 

IOSBTGXESPXHOPDE 

Decipher the message, given that it starts with the word DEAR. 


Solution 


r 9 

19 
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19 
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ri 

jo 

ri 

|o 
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ri 

|o 
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JS 
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19 
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19 

5 
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1 l^J 
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»J 
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12 
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12 
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12 

151 


7 

wj 


12 

15 
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fyvj 

12 
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17 

J 
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From Table 1, the numerical equivalent of the known plaintext is 

DE AR 
4 5 1 18 

and the numerical equivalent of the corresponding ciphertext is 

10 SB 
915 192 

so the corresponding plaintext and ciphertext vectors are 






























Cl 


We want to reduce 


f l = 9 15 

J ~ 19 2 


to I by elementary row operations and simultaneously apply these operations to 

rl 

P= 1)1 = 4 5 

J 1 13 
1 J 2 J J 

to obtain (j4 -1 ) r (the transpose of the deciphering matrix). This can be accomplished by adjoining P to 
the right of C and applying row operations to the resulting matrix [C|jP] until the left side is reduced to 

-i 7 

I. The final matrix will then have the form [I (A ) ]. The computations can be carried out as follows: 


-K r _r i o 


IA — 


Thus, 


so the deciphering matrix is 


To decipher the message, we first group the ciphertext into pairs and find the numerical equivalent of 
each letter: 


r 


915 192 207 245 1916 248 1516 45 

Next, we multiply successive ciphertext vectors on the left by a~ x and find the alphabet equivalents of 
the resulting plaintext pairs: 

1 i? ir *u 4 i d 

_0 9J|_ 15J _ |_5j B 

1 i? ir i9 i=r 1 1 a 

n i 7 ir 2 oi r 91 1 


(mod 26) 


1 171 f 24 


1 17 19 _ 5 

0 9 J |_ 1 6 J |_ 14 

1 171 f 241 f 4 


1 17 15 = 1 A 

_0 9JL 16 J _ L14J N 

"1 i 7 ir 4 i=r n i K 

_0 9 J |_ 5 J — |_ 19 J 3 
Finally, we construct the message from the plaintext pairs: 


DE AR IK ES EN DT AN K3 
DEAR IKE SEND TANKS 



Further Readings 


♦ 


Readers interested in learning more about mathematical cryptography are referred to the following books, the first of which is 
elementary and the second more advanced. 


ABRAHAM Sinkov, Elementary Cryptanalysis, a Mathematical Approach (Mathematical Association of America, 
Mathematical Library, 1966). 

2. A t. an G. Konheim, Cryptography, a Primer (NewYork: Wiley-Interscience, 1981). 


Exercise Set 11.16 



Click here for Just Ask! 


Obtain the Hill cipher of the message 
DARK NIGHT 

for each of the following enciphering matrices: 


(a) 


1 3 

2 1 


(b) 


4 3 
1 2 


In each part determine whether the matrix is invertible modulo 26. If so, find its inverse modulo 26 and check your work by 
verifying that AA~^ = A~^ A = I (mod 26). 


(a) 



1 

2 


(b) 



1 

3 


(c) 



11 

9 


(d) 



1 

7 
















Va- 3 ' 


Decode the message 

SAKNOXAOJX 

given that it is a Hill cipher with enciphering matrix 


A Hill 2-cipher is intercepted that starts with the pairs 
4 ‘ SL HK 

Find the deciphering and enciphering matrices, given that the plaintext is known to start with the word ARMY. 

Decode the following Hill 2-cipher if the last four plaintext letters are known to be ATOM. 

5 - LNGIHGYBVRENJYQO 

Decode the following Hill 3-cipher if the first nine plaintext letters are IHAVECOME : 

6 - HPAFQGGD U GDDHPGODYNOR 

All of the results of this section can be generalized to the case where the plaintext is a binary message; that is, it is a sequence of 
7. 0's and l's. In this case we do all of our modular arithmetic using modulus 2 rather than modulus 26. Thus, for example, 

1 q 1 = 0 (mod 2)- Suppose we want to encrypt the message 110101 111. Let us first break it into triplets to form the three 

_1 1 PI PI p 1 o" 

vectors 1 , 0 , 1 , and let us take 0 1 1 as our enciphering matrix. 

Oil 111 


(a) Find the encoded message. 


(b) Find the inverse modulo 2 of the enciphering matrix, and verify that it decodes your encoded message. 


If, in addition to the standard alphabet, a period, comma, and question mark were allowed, then 29 plaintext and ciphertext 
8. symbols would be available and all matrix arithmetic would be done modulo 29. Under what conditions would a matrix with 
entries in Z 29 be invertible modulo 29? 

Show that the modular equation % = 1 (mod 26) has no solution in Zjt by successively substituting the values x = 0, 1, 2,..., 25 


( a ) Let P and Cbe the matrices in Theorem 11.16.5. Show that p= CLd -1 ) 


(b) To prove Theorem 11.16.5, let ■■■’ E } 

reduce C to 7, so 


be the elementary matrices that correspond to the row operations that 




Show that 


E»‘"E 2 E l C = I 


E^E 2 B 1 P=(A~ 1 ) T 

from which it follows that the same sequence of row operations that reduces C to I converts P 
to (A- 1 /- 


11 . 

(a) If A is the enciphering matrix of a Hill //-cipher, show that 

A~ l = (C~ l P) T (mod 26) 

where C and P are the matrices defined in Theorem 11.16.5. 

(b) Instead of using Theorem 11.16.5 as in the text, find the deciphering matrix -1 of Example 8 by using the result in 
part (a) and Equation 2 to compute C -1 - 

Note Although this method is practical for Hill 2-ciphers, Theorem 11.16.5 is more efficient for Hill n-ciphers with 
n > 2 - 


Section 11.16 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Two integers that have no common factors (except 1) are said to be relatively prime. Given a positive integer n, let 

S n = {a\, a 2 , a 2 ,a m )» where a i < a 2 <<23 < < a m , be the set of all positive integers less than n and relatively prime t< 

For example, if n = 9 , then 

£ 9 = {a\,a 2 ,a 2 ,..., a^} = {1, 2,4, 5,7, 8 } 


(a) Construct a table consisting of n and for « = 2, 3,..., 15, and then compute 

m { m \ 

Y.&k and (mod«) 

k =1 Vc=l } 

in each case. Draw a conjecture for « > 15 and prove your conjecture to be true. 
Hint Use the fact that if a is relatively prime to n, then » _ a is also relatively prime to n. 

(b) Given a positive integer n and the set £ , let p n be the m x m matrix 



so that, for example, 


ai 


a 3 

... 

a 7fl — 1 

a m 

a2 

a 3 


... 

a m 

31 

a 3 


a 5 

... 

a\ 

^2 

a m— 1 


a\ 

... 

a m— 3 

2 

a m 


2 


a m—2 

a m— 1 


'l 

2 4 

5 

1 8" 



2 

4 5 

7 

8 1 


P 9 = 

4 

5 7 

8 

1 2 


5 

7 8 

1 

2 4 



7 

8 1 

2 

4 5 



8 

1 2 

4 

5 7 



Use a computer to compute det(P H ) and det(.P H )(mod>i) for« = 2, 3,..., 15, and then use these 
results to construct a conjecture. 

(c) Use the results of part (a) to prove your conjecture to be true. 

Hint Add the first m — \ rows of p n to its last row and then use Theorem 2.3.3. What do these results imply about the 
inverse ofp B ( mo d»)? 


Given a positive integer n greater than 1, the number of positive integers less than n and relatively prime to n is called the 
T2. Euler phi function of n and is denoted by _■■(«). For example, s(6') = 2 since only two positive integers (1 and 5) are less than 
6 and have no common factor with 6. 


(a) Using a computer, for each value of n = 2, 3,..., 25 compute and print out all positive integers that are less than n and 
relatively prime to n. Then use these integers to determine the values of _-{n) for m = 2, 3,.... 25. Can you discover a 
pattern in the results? 


(b) It can be shown that if p 2 , p 3 ,p m ) are all the distinct prime factors of n, then 






For example, since {2, 3} are the distinct prime factors of 12, we have 

*( ,2 > = 12(l4)M)=4 


which agrees with the fact that {1, 5, 7, 11} are the only positive integers less than 12 and 
relatively prime to 12. Using a computer, print out all the prime factors of n for n = 2, 3, 25. 
Then compute <p(n) using the formula above and compare it to your results in part (a). 


Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 





11.17 

GENETICS 


In this section we investigate the propagation of an inherited trait in successive 
generations by computing powers of a matrix. 


Eigenvalues and Eigenvectors 
Diagonalization of a Matrix 
Intuitive Understanding of Limits 


Inheritance Traits 

In this section we examine the inheritance of traits in animals or plants. The inherited trait under consideration is assumed to be 
governed by a set of two genes, which we designate by A and a. Under autosomal inheritance each individual in the population of 
either gender possesses two of these genes, the possible pairings being designated AA, Aa, and aa- This pair of genes is called the 
individual's genotype, and it determines how the trait controlled by the genes is manifested in the individual. For example, in 
snapdragons a set of two genes determines the color of the flower. Genotype AA produces red flowers, genotype At produces pink 
flowers, and genotype aa produces white flowers. In humans, eye coloration is controlled through autosomal inheritance. 
Genotypes AA and A have brown eyes, and genotype aa has blue eyes. In this case we say that gene A dominates gene a, or that 
gene a is recessive to gene A, because genotype A has the same outward trait as genotype AA- 

In addition to autosomal inheritance we will also discuss X-linked inheritance. In this type of inheritance, the male of the species 
possesses only one of the two possible genes (.A or a), and the female possesses a pair of the two genes (AA> Aa, or aa)- In humans, 
color blindness, hereditary baldness, hemophilia, and muscular dystrophy, to name a few, are traits controlled by X-linked 
inheritance. 

Below we explain the manner in which the genes of the parents are passed on to their offspring for the two types of inheritance. 

We construct matrix models that give the probable genotypes of the offspring in terms of the genotypes of the parents, and we use 
these matrix models to follow the genotype distribution of a population through successive generations. 

Autosomal Inheritance 

In autosomal inheritance an individual inherits one gene from each of its parents’ pairs of genes to form its own particular pair. As 
far as we know, it is a matter of chance which of the two genes a parent passes on to the offspring. Thus, if one parent is of 
genotype A, it is equally likely that the offspring will inherit the A gene or the a gene from that parent. If one parent is of genotype 
aa and the other parent is of genotype At, the offspring will always receive an a gene from the aa parent and will receive either an 
A gene or an a gene, with equal probability, from the A parent. Consequently, each of the offspring has equal probability of being 
genotype aa or A- In Table 1 we list the probabilities of the possible genotypes of the offspring for all possible combinations of 
the genotypes of the parents. 


Table 1 

Genotype of Offspring 

Genotypes of Parents 

AA — Ad Ad — Aa Ad — aa Aa — Aa Aa — aa aa — aa 

AA 

1 1 0 1 0 0 

2 4 






Genotype of Offspring 


Genotypes of Parents 



AA-AA 

AA — Aa 

AA — aa 

Aa — Aa 

Aa — aa 

aa — aa 

Aa 

0 

1 

1 

1 

i 

0 



2 


2 

2 


aa 

0 

0 

0 

1 

1 

1 





4 

2 



Distribution of Genotypes in a Population 


Suppose that a fanner has a large population of plants consisting of some distribution of all three possible genotypes AAAa , and 
aa • The farmer desires to undertake a breeding program in which each plant in the population is always fertilized with a plant of 
genotype AA and is then replaced by one of its offspring. We want to derive an expression for the distribution of the three possible 
genotypes in the population after any number of generations. 

For n = o, 1,2, ... , let us set 


a n = fraction of plants of genotype AA in nth generation 
= fraction of plants of genotype Aa in nth generation 
c n = fraction of plants of genotype aa in nth generation 

Thus «q, &q, and cq specify the initial distribution of the genotypes. We also have that 

a n T- &h + c yi = 1 for ft = 0, 1, 2,... 

From Table 1 we can determine the genotype distribution of each generation from the genotype distribution of the preceding 
generation by the following equations: 


a n — a-yi —i 

1 



a 

II 

a 


« = 1, 2,... 

(1) 

c n — 0 





For example, the first of these three equations states that all the offspring of a plant of genotype AA will be of genotype AA under 
this breeding program and that half of the offspring of a plant of genotype Aa will be of genotype AA- 


Equations 1 can be written in matrix notation as 


X C») = Mx (»- 1) > 


* = 1 , 2 ,... 


where 



a n 


a n — 1 

II 

byt 

c n 

ii 

7 

-1 

’— 1 i—i 

1 1 

K K 

_I 


and 


M = 


1 

2 

1 

2 

0 0 


0 


0 

1 

0 


Note that the three columns of the matrix M are the same as the first three columns of Table 1. 


(2) 


From Equation 2 it follows that 
















x V> = Mx^ = M 2 ^- 2 > = ... = M”x ( ® 


(3) 


Consequently, if we can find an explicit expression for M H , we can use 3 to obtain an explicit expression for X C»). To find an 
explicit expression for M”, we first diagonalize M. That is, we find an invertible matrix P and a diagonal matrix D such that 


M = PDP 

With such a diagonalization, we then have (see Exercise 1) 

where 


-1 


(4) 



M 

H _ 

PD n P 

-1 

for 

= 

1,2, 




Ai 

0 

0 ... 

0 ' 



0 

0 •• 

■ 0 

D H = 

0 

^2 

0 ... 

0 

= 

0 

A 2 

0 •• 

■ 0 


0 

0 

0 ... 



0 

0 

0 - 

. \H 
A k 


The diagonalization of M is accomplished by finding its eigenvalues and corresponding eigenvectors. These are as follows (verify): 


Eigenvalues: 


Ai = l, X 2 = \. A 3 = 0 



T 


1 


1 

Corresponding eigenvectors: \ T i = 

0 

V 2 = 

-1 

v 3 

-2 


0 


0 


1 


Thus, in Equation 4 we have 


D = 


Ai 0 0 

0 a 2 0 
0 0 a 3 


and 


P = t v l | v 2 |V3 ] = 


1 0 0 
o 1 0 

0 0 0 


1 1 1 

0-1 -2 
0 0 1 


Therefore, 


x (») = pD”p- 1 x (°) = 


1 1 1 

0 -1 -2 
0 0 1 



"l 

0 

o' 

r 


0 

ar 

0 




W 



- 

0 

0 

0 

- 


1 1 


1 


0 -1 -2 
0 0 1 


a 0 

Cf) 


or 


x (») = 


byt 

c n 


1 1 

o 

0 


-ft)" ‘-ft) 

ft)" 


a 0 

^0 

CQ 


flO + ^O+co- *0“ C Q 

*)"*■ - ft)""- 


Using the fact that a0 | | = l, we thus have 



co n = 1 , 2 ,... 


( 5 ) 


c n — 0 


These are explicit formulas for the fractions of the three genotypes in the nth generation of plants in terms of the initial genotype 
fractions. 

Because j tends to zero as n approaches infinity, it follows from these equations that 

a n - 1 

byi -► 0 

c n = 0 

as n approaches infinity. That is, in the limit all plants in the population will be genotype Ad- 


Modifying Example 1 


We can modify Example 1 so that instead of each plant being fertilized with one of genotype AA, each plant is fertilized with a 
plant of its own genotype. Using the same notation as in Example 1, we then find 

= M”x ® 

where 

i 0 

M= 0 | 0 
0 ^ 1 

The columns of this new matrix M are the same as the columns of Table 1 corresponding to parents with genotypes AA — AA 
Aa — Aa, and act — act- 


The eigenvalues of M are (verify) 


Ai = l, A 2 = l, A 3 = - 


The eigenvalue Aj = 1 has multiplicity two and its corresponding eigenspace is two-dimensional. Picking two linearly independent 
eigenvectors vq and v 2 in that eigenspace, and a single eigenvector v 3 for the simple eigenvalue a 3 = -i, we have (verify) 



V 


'o' 


1 

VI = 

0 

v 2 = 

0 

v 3 = 

-2 


0 


1 


1 


The calculations for x (^ are then 












Thus, 


= M”jP> = PD n P~ l x^ 



i—«■ 

o 

o 

"10 r 



0 1 0 

0 0-2 

0 1 1 

0 0 (If 


W J 
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2 

1 

2 

1 

2 


0 

1 

0 


a 0 

^0 


co 



In the limit, as n tends to 


infinity, 


0 and fi'i 


H + l 


a n 

b n 

c n 


0, so 

fl 0 + ^0 
o 

CO + ^0 


( 6 ) 


Thus, fertilization of each plant with one of its own genotype produces a population that in the limit contains only genotypes AA 
and aa- 


Autosomal Recessive Diseases 

There are many genetic diseases governed by autosomal inheritance in which a normal gene A dominates an abnormal gene a. 
Genotype AA is a normal individual; genotype Aa is a carrier of the disease but is not afflicted with the disease; and genotype aa is 
afflicted with the disease. In humans such genetic diseases are often associated with a particular racial group—for instance, cystic 
fibrosis (predominant among Caucasians), sickle-cell anemia (predominant among blacks), Cooley's anemia (predominant among 
people of Mediterranean origin), and Tay-Sachs disease (predominant among Eastern European Jews). 

Suppose that an animal breeder has a population of animals that carries an autosomal recessive disease. Suppose further that those 
animals afflicted with the disease do not survive to maturity. One possible way to control such a disease is for the breeder to 
always mate a female, regardless of her genotype, with a normal male. In this way, all future offspring will either have a normal 
father and a normal mother (AA — AA matings) or a normal father and a carrier mother (AA — Aa matings). There can be no 
AA — aa matings since animals of genotype a a do not survive to maturity. Under this type of mating program no future offspring 
will be afflicted with the disease, although there will still be carriers in future generations. Let us now determine the fraction of 
carriers in future generations. We set 


where 






















a yi = fraction of population of genotype AA in nth generation 

b n = fraction of population of genotype Act (carriers) in nth generation 


Because each offspring has at least one normal parent, we may consider the controlled mating program as one of continual mating 
with genotype AA, as in Example 1. Thus, the transition of genotype distributions from one generation to the next is governed by 
the equation 

= * = 1,2,... 


where 



1 

2 

1 

2 


Because we know the initial distribution the distribution of genotypes in the nth generation is thus given by 

x(”) = M n x®, *=1,2,... 


The diagonalization of M is easily carried out (see Exercise 4) and leads to 


x®>=PD n p- 1 xf® = 



'1 0 




'1 1 
_0 - 1_ 

0 (2)" 

'1 

_0 

1 

-1_ 

"ao" 

^0 


w J 



“ 


1 

0 



Because = ], we have 



(7) 


Thus, as n tends to infinity, we have 

a n —i 1 

b-n —> 0 

so in the limit there will be no carriers in the population. 


From 7 we see that 



*= 1 , 2 ,... 


( 8 ) 


That is, the fraction of carriers in each generation is one-half the fraction of carriers in the preceding generation. It would be of 
interest also to investigate the propagation of carriers under random mating, when two animals mate without regard to their 
genotypes. Unfortunately, such random mating leads to nonlinear equations, and the techniques of this section are not applicable. 
However, by other techniques it can be shown that under random mating, Equation 8 is replaced by 


b n — ■ 


3 h-1 


1H jb n -l 


* = 1 , 2 , 


(9) 


As a numerical example, suppose that the breeder starts with a population in which 10% of the animals are carriers. Under the 
controlled-mating program governed by Equation 8, the percentage of carriers can be reduced to 5% in one generation. But under 
random mating, Equation 9 predicts that 9.5% of the population will be carriers after one generation (b = 0.95 if b n -\ = 10)- In 
addition, under controlled mating no offspring will ever be afflicted with the disease, but with random mating it can be shown that 
about 1 in 400 offspring will be bom with the disease when 10% of the population are carriers. 


X-Linked Inheritance 



















As mentioned in the introduction, in X-linked inheritance the male possesses one gene (A or a) and the female possesses two genes 
(AA, Aa , or aa)- The term X-linked is used because such genes are found on the X-chromosome, of which the male has one and the 
female has two. The inheritance of such genes is as follows: A male offspring receives one of his mother's two genes with equal 
probability, and a female offspring receives the one gene of her father and one of her mother's two genes with equal probability. 
Readers familiar with basic probability can verify that this type of inheritance leads to the genotype probabilities in Table 2. 

Table 2 



Genotypes nf Parents (Father, Mother) 

[A»AA l 

iA . Aa) 

lA, utt) 


(a m Aa 1 

Iff, ffffJ 

Offspring 

fcH- 

A 

i 

1 

2 

0 

i 

2 

0 

a 

d 

j 

1 

0 

2 

1 

a 

JB 

AA 

i 


0 

0 

0 

0 

Aa 

0 


! 

i 

2 

0 

m 

0 

0 

0 

0 

1 

1 


We will discuss a program of inbreeding in connection with X-linked inheritance. We begin with a male and female; select two of 
their offspring at random, one of each gender, and mate them; select two of the resulting offspring and mate them; and so forth. 
Such inbreeding is commonly performed with animals. (Among humans, such brother-sister marriages were used by the rulers of 
ancient Egypt to keep the royal line pure.) 

The original male-female pair can be one of the six types, corresponding to the six columns of Table 2: 

(A, AA), (A, Aa), (A,aa), (a, AA), (a,Aa), ( a,aa) 

The sibling pairs mated in each successive generation have certain probabilities of being one of these six types. To compute these 
probabilities, for n = 0, 1,2, ..., let us set 

a n = probability sibling-pair mated in nth generation is type (A, AA) 

= probability sibling-pair mated in nth generation is type (A, Aa) 
c n = probability sibling-pair mated in nth generation is type (A, a a) 
d n = probability sibling-pair mated in nth generation is type (a, AA) 
e n = probability sibling-pair mated in nth generation is type (a. As) 
f = probability sibling-pair mated in nth generation is type (a, act) 


With these probabilities we form a column vector 


x 00 


i mm Table 2 it follows that 


- H 

c n 

dyt 

In 


« = 0 , 1 , 2 , 


x (») = 1) 


n = 1 , 2 ,... 


( 10 ) 


where 






















M - 


(A, Ad) (A, /Iff) (A,{Ui) (i'f. A A > {a T Att) tit. aa) 

I J 0 0 0 0 "I (A,AA) 

0 ^ 01^0 (A, Act) 

0 0 0 0 \ 0 ( A,aa ) 

0 1 0 0 0 0 (w. A A) 

0 J 1 0 \ 0 (a . A«) 

_ G 0 0 0 J 1 J («,««) 

For example, suppose that in the { n — l)-st generation, the sibling pair mated is type (A, Aa)- Then their male offspring will be 
genotype A or a with equal probability, and their female offspring will be genotype AA or Aa with equal probability. Because one 
of the male offspring and one of the female offspring are chosen at random for mating, the next sibling pair will be one of type 

(A, AA )> (A, Aa )> (a, AA), or Aa) with equal probability. Thus, the second column of M contains “-j-” in each of the four rows 
corresponding to these four sibling pairs. (See Exercise 9 for the remaining columns.) 


As in our previous examples, it follows from 10 that 

x (») = ^» x (0;i „ = i >2> ... 

After lengthy calculations, the eigenvalues and eigenvectors of M turn out to be 

A 1 = l, A 2 = l, A 3 = i A 4 =--i Aj = i(l + /5), A 6 = i(l-/5) 



The diagonalization of M then leads to 


( 11 ) 


( 12 ) 


where 




1 0 
0 0 
0 0 

0 0 
0 0 
0 1 


’ 4 

-3-/5) 

? ( - 3 

+ /5) 

-6 

1 

1 


- 3 

-Uf5) 



3 ?<■ 

-1 + /5) 


-f5) 

6 

1 

1 


-> 4 

-3-/5) 

?<- 3 

4/5) 



p~ { 


1 

0 

0 

0 

0 

0 


24 


12 12 24 

i/5 i(5 + /5) 0 


We will not write out the matrix product in 12, as it is rather unwieldy. However, if a specific vector X (P) is given, the calculation 
for S ( H ) is not too cumbersome (see Exercise 6). 


Because the absolute values of the last four diagonal entries of D are less than 1, we see that as n tends to infinity, 

”1 0 0 0 0 0 “ 

0 1 0 0 0 0 

D n 000000 

0 0 0 0 0 0 

0 0 0 0 0 0 

0 0 0 0 0 0 


And so, from Equation 12, 


P 


1 0 0 
0 1 0 
0 0 0 
0 0 0 
0 0 0 
0 0 0 


0 0 
0 0 
0 0 
0 0 
0 0 
0 0 


0 

0 


0 

0 




Performing the matrix multiplication on the right, we obtain (verify) 























(13) 


r (») 


^0 4 I ‘^' c 0 4 0 3” 0 




0 

0 

0 

0 


That is, in the limit all sibling pairs will be either type (A, AA) or type (a, aa)- For example, if the initial parents are type (A, Aa) 
(that is, £ 0 = i and fl0 = C[l = = e[l = f Q = o), then as n tends to infinity, 


x 


(") 


2 

3 

0 

0 

0 

0 

1 

3 


o 1 

Thus, in the limit there is probability ^ that the sibling pairs will be (A, AA), and probability 4. that they will be (a, aa)- 


Exercise Set 11.17 



Click here for Just Ask! 


Show that if _ ppp -1, then m n = P£> n P -1 for n = 1, 2,.... 


In Example 1 suppose that the plants are always fertilized with a plant of genotype Aa rather than one of genotype AA- Derive 
2. formulas for the fractions of the plants of genotypes AA, Aa, and aa in the nth generation. Also, find the limiting genotype 
distribution as n tends to infinity. 

In Example 1 suppose that the initial plants are fertilized with genotype AA, the first generation is fertilized with genotype Aa, 

3* the second generation is fertilized with genotype AA, and this alternating pattern of fertilization is kept up. Find formulas for the 
fractions of the plants of genotypes AA, Aa, and a a in the nth generation. 

In the section on autosomal recessive diseases, find the eigenvalues and eigenvectors of the matrix M and verify Equation 7. 

4. 

Suppose that a breeder has an animal population in which 25% of the population are carriers of an autosomal recessive disease. 

5. If the breeder allows the animals to mate irrespective of their genotype, use Equation 9 to calculate the number of generations 
required for the percentage of carriers to fall from 25% to 10%. If the breeder instead implements the controlled-mating 
program determined by Equation 8, what will the percentage of carriers be after the same number of generations? 

In the section on X-linked inheritance, suppose that the initial parents are equally likely to be of any of the six possible 
genotype parents; that is, 


6 . 








X 


<p> = 


1 

6 

1 

6 

1 

6 

1 

6 

1 

6 

1 

6 


Using Equation 12, calculate 1 and also calculate the limit of 1 as n tends to infinity. 


From 13 show that under X-linked inheritance with inbreeding, the probability that the limiting sibling pairs will be of type 
7- (A, AA) is the same as the proportion of A genes in the initial population. 


In X-linked inheritance suppose that none of the females of genotype As survive to maturity. Under inbreeding the possible 
8* sibling pairs are then 

(A, AA), (A,aa), (a,AA), and (a, aa) 

Find the transition matrix that describes how the genotype distribution changes in one generation. 


Derive the matrix M in Equation 10 from Table 2. 

9. 


Section 11.17 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Tl. 


(a) Use a computer to verify that the eigenvalues and eigenvectors of 





M = 


as given in the text are correct. 




0 


1 


0 0 0 
0 1 


0 0 0 0 


1 

4 

1 


0-^000 

4 

0 4- 1 0 4- 


00004 

4 


0 

0 

0 

0 

0 

1 


(b) Starting with > and the assumption that 


Hm x^ = x 
n —* oo 

exists, we must have 

Hm i ( m) = M Hm or x = Mx 

n —* oo h —* oo 

This suggests that x can be solved directly using the equation (iif-/)x=o. Use a computer to 
solve the equation x = Mx, where 

a 

b 

c 

x= d 
e 

f 

and a 4 b + c i d-\-e i / = 1; compare your results to Equation 13. Explain why the solution to 
(M-l)x = 0 along with a4i+c4^+e4/ = lis not specific enough to determine Jim x^\ 


T2. 


(a) Given 


1 0 
0 0 
0 0 

0 0 
0 0 
0 1 


-1 1 {(-3-/5) {(-3+/5) 

2-6 1 1 

-1 -3 {(-1 I /5) {(-1-/5) 

1 3 {(-1 + /5) {(-1-/5) 

-2 6 1 1 

1 _i {(-3-/5) {(-3+/5) 


Hm 

H—*00 


1 0 0 0 0 0 
0 1 0 0 0 0 
0 0 0 0 0 0 
0 0 0 0 0 0 
0 0 0 0 0 0 
0 0 0 0 0 0 


from Equation 12 and 


















use a computer to show that 


lim M n 

H—J-OO 


0 0 
0 0 
0 0 
0 0 


1 2 

3 3 

0 0 
0 0 
0 0 
0 0 

2 1 

3 3 


0 0 
0 0 
0 0 
0 0 


(b) Use a computer to calculate M n for « = 10, 20, 30, 40, 50, 60, 70, and then compare your results to the limit in part 
(a). 


Copyright © 2005 John Wiley & Sons, Inc. All rights reserved. 





11.18 

AGE-SPECIFIC 
POPULATION GROWTH 


In this section we investigate, using the Leslie matrix model, the growth over 
time of a female population that is divided into age classes. We then determine 
the limiting age distribution and growth rate of the population. 


Eigenvalues and Eigenvectors 
Diagonalization of a Matrix 
Intuitive Understanding of Limits 

One of the most common models of population growth used by demographers is the so-called Leslie model developed in the 
1940s. This model describes the growth of the female portion of a human or animal population. In this model the females are 
divided into age classes of equal duration. To be specific, suppose that the maximum age attained by any female in the population 
is L years (or some other time unit) and we divide the population into n age classes. Then each class is Lin years in duration. We 
label the age classes according to Table 1. 


Table 1 


Age Class 

Age Interval 

1 

[0 ,Lin\ 

2 

[Lin, 2 Lin] 

3 

[2 Lin, 3 Lin] 

ft — 1 

[{n-2)L!n, {n-\)Lin} 

n 

[( n-\)Lin,L] 


Suppose that we know the number of females in each of the n classes at time ? = 0- In particular, let there be females in the 

first class, females in the second class, and so forth. With these n numbers we form a column vector: 

*2 


x 


m = 



We call this vector the initial age distribution vector. 



As time progresses, the number of females within each of the n classes changes because of three biological processes: birth, death, 
and aging. By describing these three processes quantitatively, we will see how to project the initial age distribution vector into the 
future. 


The easiest way to study the aging process is to observe the population at discrete times—say, t\, ty- > The Leslie 







model requires that the duration between any two successive observation times be the same as the duration of the age intervals. 
Therefore, we set 

^0 = 0 
t\ — L i n 
tj = 2L / n 

tk = kLi n 

With this assumption, all females in the (j | i)-st class at time ^ f j were in the z'th class at time 

The birth and death processes between two successive observation times can be described by means of the following demographic 
parameters: 

&i The average number of daughters born to each female during the time she is in the z'th age class 

(i = 1.2.») 

The fraction of females in the z'th age class that can be expected to survive and pass into the (j f l)-st age 

(j = 1, 2,— 1) class 

By their definitions, we have that 


0 ) a< > 0 for j = 1 , 2 , ..., n 


(ii) 0 <bj < 1 for i = 1 , 2 , 1 

Note that we do not allow any £. to equal zero, because then no females would survive beyond the z'th age class. We also assume 
that at least one a,- is positive so that some births occur. Any age class for which the corresponding value of a,- is positive is called 
a fertile age class. 


We next define the age distribution vector X ( A) at time t k by 


M _ 


(k) 

r l 

.(*) 


(*) 


where Jfi) is the number of females in the 

x i 

daughters born between times t' K _\ an d t% 

, ( 


L. _J 

5 class at time t k . Now, at time the females in the first age class are just 


or, mathematically, 


^number of y 


I females in 

> = { 

\ class 1 at 

time tir 
\ * ) 


\ 


number of 
daughters bom 


number of 
daughters bom 


number of 


class 1 between class 21 _ 

timesand£fc^ primes and^ 


daughters bom 

in \ I —h I to females in ^ 
class« between 
times tk—i and^ 


The females in the (j 4 l)-st age class (j 
alive at time ? k . Thus, 


l —^1-^1 I a 2^2 "" I 

i = 1 , 2 ,..., n — 1 ) at time t k are those females in the z'th class at time who ; 







or, mathematically, 


'number of ^ 

( 

females in 

> = < 

\ class i + 1 at 

time tk 


\ / 

. 


fraction of 
females in 
class i who 
survive and pass 

into class i 4- 1 




'number of'' 
females in 
class i at 
time 


..W 


- !+ i =v, 


Cfc-i) 


i = 1, 2,.... n — 1 


( 2 ) 


Using matrix notation, we can write Equations 1 and 2 as 


~t*)~ 


'31 


3 3 - 

" a n— 1 

3 h 

*1 

a) 

x 2 


h 

0 

0 • 

0 

0 

to 

1 

1- L 

(k) 

= 

0 

h 

0 ■ 

0 

0 

(*-l) 

x 3 


: 




: 

x 3 

*3 - 


0 

0 

0 ■ 

'■ b Yl — 1 

0 

1 - 

* 

i 

1- L 


or, more compactly, 


where L is the Leslie matrix 


From Equation 3 it follows that 




£= 1 , 2 ,... 


a i 32 


Z = 


*1 0 0 
0 b 2 o 


1 a n 
0 0 
0 0 


000- b „-1 0 


x © =ix CD = i 2 x © 

x^=Ix^ = I 3 ^ 

v^ = /.v^- 1 ) = />vC°) 


(3) 


(4) 


(5) 


Thus, if we know the initial age distribution x — 1 and the Leslie matrix L, we can determine the female age distribution at any later 
time. 


Female Age Distribution for Animals 


Suppose that the oldest age attained by the females in a certain animal population is 15 years and we divide the population into 
three age classes with equal durations of five years. Let the Leslie matrix for this population be 



4 3 
0 0 


If there are initially 1000 females in each of the three age classes, then from Equation 3 we have 


























x^ = 


1,000 

1,000 

1,000 



1 

2 

0 

0 

1,000 


"7,000" 

& = £jF> = 

1,000 

= 

500 


0 

1 

4 

0 

1,000 


250 


"o 

4 

3‘ 





1 

2 

0 

0 

"7,000" 


"2,750" 

& = £j&> = 

500 

= 

3,500 


0 

1 

4 

0 

250 


125 


'o 

4 

3" 





1 

2 

0 

0 

"2,750" 


"14,375 

x ^ = Zx © = 

3,500 

= 

1,375 


0 

1 

0 

125 


875 


4 





Thus, after 15 years there are 14,375 females between 0 and 5 years of age, 1375 females between 5 and 10 years of age, and 875 
females between 10 and 15 years of age. 


Limiting Behavior 

Although Equation 5 gives the age distribution of the population at any time, it does not immediately give a general picture of the 
dynamics of the growth process. For this we need to investigate the eigenvalues and eigenvectors of the Leslie matrix. The 
eigenvalues of L are the roots of its characteristic polynomial. As we ask the reader to verify in Exercise 2, this characteristic 
polynomial is 

p(X) = \\l~L\ 

= A M —fljA” a2&iA M ^ — ct2 s b\b2^ yi ^- a n b\b2"‘b yi —\ 

To analyze the roots of this polynomial, it will be convenient to introduce the function 

qW - A f a 2 4 A 3 4 4 A » (6) 

Using this function, the characteristic equation p (A) = 0 can be written (verify) 

q(X) = 1 forA^O ^ 

Because all the a ,• and £. are nonnegative, we see that q (A) is monotonically decreasing for A greater than zero. Furthermore, q (A) 

has a vertical asymptote at A = 0 and approaches zero as \ _* rvn. Consequently, as Figure 11.18.1 indicates, there is a unique A, 

say A = Ap such that ^(Aj) = 1 - That is, the matrix L has a unique positive eigenvalue. It can also be shown (see Exercise 3) that 
Aj has multiplicity 1; that is, Aj is not a repeated root of the characteristic equation. Although we omit the computational details, 
the reader can verify that an eigenvector corresponding to Aj is 

1 

*1 /Ai 


xi 


bib 2 t\- 


bi b 2 b 3 /X( 


b\b2-byi-i i X 


H —1 


( 8 ) 


Because Ai has multiplicity 1, its corresponding eigenspace has dimension one (Exercise 3), and so any eigenvector corresponding 




























to it is some multiple of x j. We can summarize these results in the following theorem. 



THEOREM 11.18.1 


Existence of a Positive Eigenvalue 

A Leslie matrix L has a unique positive eigenvalue Ap This eigenvalue has multiplicity 1 and an eigenvector xi all of whose 
entries are positive. 


We will now show that the long-term behavior of the age distribution of the population is determined by the positive eigenvalue Aj 
and its eigenvector xp 

In Exercise 9 we ask the reader to prove the following result. 


THEOREM 11.18.2 


Eigenvalues of a Leslie Matrix 

If X i is the unique positive eigenvalue of a Leslie matrix L, and \ k is any other real or complex eigenvalue ofL, then |A^| < Ap 


For our purposes the conclusion in Theorem 11.18.2 is not strong enough; we need Aj to satisfy |A^| < Aj. In this case Aj would be 
called the dominant eigenvalue of L. However, as the following example shows, not all Leslie matrices satisfy this condition. 


Leslie Matrix with No Dominant Eigenvalue 


Let 


Then the characteristic polynomial of L is 



0 6 
0 0 


P( A) = 


XI-L 


= A 3 -1 


The eigenvalues of L are thus the solutions of a 3 = 1—namely, 












A=l, 


1 + J^ _1_J^ 

2 2 ' 2 2 

All three eigenvalues have absolute value 1, so the unique positive eigenvalue Aj = 1 is not dominant. Note that this matrix has the 
property that = /. This means that for any choice of the initial age distribution we have 

x©=x® = x® =.= x^ = - 

The age distribution vector thus oscillates with a period of three time units. Such oscillations (or population waves, as they are 
called) could not occur if A^ were dominant, as we will see below. 


It is beyond the scope of this book to discuss necessary and sufficient conditions for a, to be a dominant eigenvalue. However, we 
will state the following sufficient condition without proof. 


THEOREM 11.18.3 


Dominant Eigenvalue 

If two successive entries a,- and in the first row of a Leslie matrix L are nonzero, then the positive eigenvalue ofL is 
dominant. 


Thus, if the female population has two successive fertile age classes, then its Leslie matrix has a dominant eigenvalue. This is 
always the case for realistic populations if the duration of the age classes is sufficiently small. Note that in Example 2 there is only 
one fertile age class (the third), so the condition of Theorem 11.18.3 is not satisfied. In what follows, we always assume that the 
condition of Theorem 11.18.3 is satisfied. 


Let us assume that L is diagonalizable. This is not really necessary for the conclusions we will draw, but it does simplify the 
arguments. In this case, L has n eigenvalues, Ap A 2 , •••> A H , not necessarily distinct, and it linearly independent eigenvectors, xpx 2 , 
..., x H , corresponding to them. In this listing we place the dominant eigenvalue Aj first. We construct a matrix P whose columns 
are the eigenvectors of L: 

P= [xi|x 2 |x 3 |-|x h ] 


The diagonalization of L is then given by the equation 


L=P 


From this it follows that 


L k = P 


Ai 0 0 

0 a 2 0 
0 0 0 


Aj 0 0 

o aJ o 


0 0 0 


0 

0 

a h 




Aj 0 0 

0 a} 0 


,-l 


for k = 1, 2, .... For any initial age distribution vector x (°), we then have 


0 0 0 - K 


P-'x® 


for k = 1, 2, .... Dividing both sides of this equation by a* and using the fact that x (^ = we have 









J-v(*)-P 

,k ~ 


ft) 


0 0 . 0 

k 

0 . 0 


0 0 


t) 


p-! x © 


(9) 


Because Aj is the dominant eigenvalue, we have |A,- / Ai | < 1 for j = 2 , 3, It follows that 


(VAi V 


0 as k 


do forz = 2, 3,m 


Using this fact, we may take the limit of both sides of 9 to obtain 

1 0 0 - o' 


lim , . 
k + oo| A* 




= P 


0 0 0 -0 


p- l ^> 


( 10 ) 


0 0 0 -0 

Let us denote the first entry of the column vector by the constant c. As we ask the reader to show in Exercise 4, the right 

side of 10 can be written as cxj, where c is a positive constant that depends only on the initial age distribution vector x ®. Thus, 10 
becomes 


lim 


°° a; 


'-M* =«, 


Equation 11 gives us the approximation 


for large values of k. From 12 we also have 


Comparing Equations 12 and 13, we see that 


x (£) ^ cA^xi 




x^A^*- 1 ) 


( 11 ) 


( 12 ) 


(13) 


(14) 


for large values of k. This means that for large values of time, each age distribution vector is a scalar multiple of the preceding age 
distribution vector, the scalar being the positive eigenvalue of the Leslie matrix. Consequently, the proportion of females in each 
of the age classes becomes constant. As we will see in the following example, these limiting proportions can be determined from 
the eigenvector x i. 


Example 1 Revisited 


The Leslie matrix in Example 1 was 


L = 


0 4 

1 
2 


i 0 


0 i 


3 

0 

0 


Its characteristic polynomial is p (A) 
corresponding eigenvector xi is 


= A 3 — 2A — J-, and the reader can verify that the positive eigenvalue is Ai = —• From 8 the 
8 2 











X! = 


1 

Al/Ai 
*1*2/A? 


1 

1 

2 . 

3 

2 


l Vl 


2 A4 



1 

1 

3 

J_ 

18 


From 14 we have 

V*) - 

for large values of k. Hence, every five years the number of females in each of the three classes will increase by about 50%, as will 
the total number of females in the population. 


From 12 we have 


x 


m 



i 

l 

3 

j_ 

18 


Consequently, eventually the females will be distributed among the three age classes in the ratios 1 : 

distribution of 72% of the females in the first age class, 24% of the females in the second age class, 
third age class. 


■j: -jL. This corresponds to a 


and 4% of the females in the 


♦ 


EXAMPLE 4 Female Age Distribution for Humans 


In this example we use birth and death parameters from the year 1965 for Canadian females. Because few women over 50 years of 
age bear children, we restrict ourselves to the portion of the female population between 0 and 50 years of age. The data are for 
5-year age classes, so there are a total of 10 age classes. Rather than writing out the 10x10 Leslie matrix in full, we list the birth 
and death parameters as follows: 


Age Interval 

a. 

*.- 

[0, 5) 

0.00000 

0.99651 

[5, 10 ) 

0.00024 

0.99820 

[10, 15) 

0.05861 

0.99802 

[15, 20) 

0.28608 

0.99729 

[20, 25) 

0.44791 

0.99694 

[25, 30) 

0.36399 

0.99621 

[30, 35) 

0.22259 

0.99460 

[35, 40) 

0.10457 

0.99184 



















Age Interval 

Cti 

bi 

[40, 45) 

0.02826 

0.98700 

[45, 50) 

0.00240 

— 


Using numerical techniques, we can approximate the positive eigenvalue and corresponding eigenvector by 


Ai = 1.07622 


and 


1.00000 

0.92594 

0.85881 

0.79641 

0.73800 

0.68364 

0.63281 

0.58482 

0.53897 

0.49429 


Thus, if Canadian women continued to reproduce and die as they did in 1965, eventually every 5 years their numbers would 
increase by 7.622%. From the eigenvector xp we see that, in the limit, for every 100,000 females between 0 and 5 years of age, 
there will be 92,594 females between 5 and 10 years of age, 85,881 females between 10 and 15 years of age, and so forth. 

♦ 


Let us look again at Equation 12, which gives the age distribution vector of the population for large times: 

x (fc) ~ X1 

Three cases arise according to the value of the positive eigenvalue Ap 


(i) The population is eventually increasing if Aj > 1- 

(ii) The population is eventually decreasing if Aj < 1- 

(iii) The population eventually stabilizes if Aj = 1- 

The case A^ = 1 is particularly interesting because it determines a population that has zero population growth. For any initial age 
distribution, the population approaches a limiting age distribution that is some multiple of the eigenvector xp From Equations 6 
and 7, we see that Aj = 1 is an eigenvalue if and only if 

a\ +ct 2 b\ -V 4-f a n b\b 2 “b n -\ = 1 nf - 


The expression 


R = a l I -ajb\ 4- a 2 ,b\b 2 H-P ct yi b\b 2 "‘by l —\ 

is called the net reproduction rate of the population. (See Exercise 5 for a demographic interpretation of R.) Thus, we can say that 
a population has zero population growth if and only if its net reproduction rate is 1. 


Exercise Set 11.18 








1 . 


Suppose that a certain animal population is divided into two age classes and has a Leslie matrix 


L = 


' I 
2 ° 


(a) Calculate the positive eigenvalue Aj of L and the corresponding eigenvector jq. 


(b) Beginning with the initial age distribution vector 


... 


100 

0 


calculate x 1 ^, x ( ' 4) , and rounding off to the nearest integer when necessary. 


(c) Calculate using the exact formula x ( 6:i = and using the approximation formula ^ Aix^- 


2 . 


Find the characteristic polynomial of a general Leslie matrix given by Equation 4. 


3. 


(a) Show that the positive eigenvalue Ai of a Leslie matrix is always simple. Recall that a root Ao of a polynomial q (A) is 
simple if and only if ^(Aq) * 0 . 


(b) Show that the eigenspace corresponding to Aj has dimension 1. 


Show that the right side of Equation 10 is cxj, where c is the first entry of the column vector j p _1 x ( l " l ' ) - 

4. 

Show that the net reproduction rate R, defined by 17, can be interpreted as the average number of daughters born to a single 
5- female during her expected lifetime. 

Show that a population is eventually decreasing if and only if its net reproduction rate is less than 1. Similarly, show that a 
6* population is eventually increasing if and only if its net reproduction rate is greater than 1. 

Calculate the net reproduction rate of the animal population in Example 1. 

7. 


8. (For Readers With a Hand Calculator) Calculate the net reproduction rate of the Canadian female population in Example 4. 


9. 


(For Readers Who Have Read Sections 10.1-10.3) Prove Theorem 11.18.2. 







Hint Write \ k = re 1 ®’ substitute into 7, take the real parts of both sides, and show that r<X\- 


Section 11.18 


Technology Exercises 

The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Tl. 


Consider the sequence of Leslie matrices 


L 2 = 


l 4 = 




' 0 

0 

a 

0 a 
b\ 0 

. £3 = 

it 

0 

0 

1 


0 

b 2 

0 







' 0 

0 

0 

0 

a 

' 0 

0 

0 

a 


it 

0 

0 

0 

0 

*1 

0 

0 

0 


b 2 

0 

0 

0 

0 

bj 

0 

0 

> L 5 = 

0 


0 

0 

hi 

0 

0 

0 

0 

£3 

0 









0 

0 

0 

b 4 

0 


(a) Use a computer to show that 


4 = h, 


4 = h. 


4=i 4 , 


4=1 5. 


for a suitable choice of a in terms of b\, b 2 r ■■■/ b n ~i- 

(b) From your results in part (a), conjecture a relationship between a and b\,b^ •••, that will make = / , where 


L n — 


’ 0 

0 

0 

0 

a 

bi 

0 

0 

0 

0 

0 

b 2 

0 

... 0 

0 

0 

0 

b 2 

... 0 

0 

0 

0 

0 

■■■ b n —\ 

0 


(c) Determine an expression for p n (X) = |A/„ — L „| and use it to show that all eigenvalues of L n satisfy |A| = 1 when a 
and b\> b 2 > •••> b n -\ are related by the equation determined in part (b). 


T2. 


Consider the sequence of Leslie matrices 

















where 0<;<l,0<kl and 1 < a - 


(a) Choose a value for n (say, n = g). For various values of a, b, and p, use a computer to determine the dominant 
eigenvalue of £ , and then compare your results to the value of a +- bp. 


(b) Show that 


which means that the eigenvalues of L n must satisfy 

A M+1 -(a-{ bp)X n + a (bp) ” = 0 

(c) Can you now provide a rough proof to explain the fact that Aj s a -\ bp ? 



Suppose that a population of mice has a Leslie matrix L over a 1-month period and an initial age distribution vector gix 

T3. by 



(a) Compute the net reproduction rate of the population. 














(b) Compute the age distribution vector after 100 months and 101 months, and show that the vector after 101 weeks is 
approximately a scalar multiple of the vector after 100 months. 

(c) Compute the dominant eigenvalue of L and its corresponding eigenvector. How are they related to your results in part 
(b)? 

(d) Suppose you wish to control the mouse population by feeding it a substance that decreases its age-specific birthrates 
(the entries in the first row of L ) by a constant fraction. What range of fractions would cause the population eventually 
to decrease? 
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11.19 

HARVESTING OF ANIMAL 
POPULATIONS 


In this section we employ the Leslie matrix model of population growth to model 
the sustainable harvesting of an animal population. We also examine the effect 
of harvesting different fractions of different age groups. 


Age-specific Population Growth (Section 11.18) 


Harvesting 

In Section 11.18 we used the Leslie matrix model to examine the growth of a female population that was divided into discrete age 
classes. In this section, we investigate the effects of harvesting an animal population growing according to such a model. By 
harvesting we mean the removal of animals from the population. (The word harvesting is not necessarily a euphemism for 
“slaughtering”; the animals may be removed from the population for other purposes.) 

In this section we restrict ourselves to sustainable harvesting policies. By this we mean the following: 


DEFINITION 


A harvesting policy in which an animal population is periodically harvested is said to be sustainable if the yield of each harvest 
is the same and the age distribution of the population remaining after each harvest is the same. 

Thus, the animal population is not depleted by a sustainable harvesting policy; only the excess growth is removed. 


As in Section 11.18, we will discuss only the females of the population. If the number of males in each age class is equal to the 
number of females—a reasonable assumption for many populations—then our harvesting policies will also apply to the male 
portion of the population. 


The Harvesting Model 


Figure 11.19.1 illustrates the basic idea of the model. We begin with a population having a particular age distribution. It undergoes 
a growth period that will be described by the Leslie matrix. At the end of the growth period, a certain fraction of each age class is 
harvested in such a way that the unharvested population has the same age distribution as the original population. This cycle repeats 
after each harvest so that the yield is sustainable. The duration of the harvest is assumed to be short in comparison with the growth 
period so that any growth or change in the population during the harvest period can be neglected. 


Population before growth period 


Population aftergrowth period 
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Figure 11.19.1 


To describe this harvesting model mathematically, let 


x = 


*1 
*2 


be the age distribution vector of the population at the beginning of the growth period. Thus x 3 - is the number of females in the /th 
class left unharvested. As in Section 11.18, we require that the duration of each age class be identical with the duration of the 
growth period. For example, if the population is harvested once a year, then the population is divided into 1-year age classes. 

If L is the Leslie matrix describing the growth of the population, then the vector Lx is the age distribution vector of the population 
at the end of the growth period, immediately before the periodic harvest. Let ^for j = ], 2,..., n, be the fraction of females from 
the /th class that is harvested. We use these n numbers to form an n x n diagonal matrix 


H = 


hi 

0 

0 


0 

h'l 

0 


0 

0 


0 0 0 


0 

0 

0 

hyi 


which we shall call the harvesting matrix. By definition, we have 

0<^; <1 (i = 1, 2 ,m) 

That is, we may harvest none (k i = 0), all (k i = 1), or some fraction (0 < hi < 1) of each of the n classes. Because the number of 
females in the /th class immediately before each harvest is the /th entry (Lx) , of the vector Lx, the /th entry of the column vector 


HLx = 

is the number of females harvested from the /th class. 

From the definition of a sustainable harvesting policy, we have 


h2(lx)2 

hyi(.Lx)n 


age distribution 


age distribution 

at end of 

— [harvest] = 

at beginning of 

growth period 


growth period 


or, mathematically, 


Lx — HLx = x 


If we write Equation 1 in the form 


(I - H)Lx = x 


( 1 ) 

( 2 ) 


we see that x must be an eigenvector of the matrix (/ _ H")L corresponding to the eigenvalue 1. As we will now show, this places 
certain restrictions on the values of and x. 


Suppose that the Leslie matrix of the population is 












Then the matrix (/ — H)L is (verify) 


(I-H)L = 


(l-Al)fll (1 — Ai)< 32 (l-^l)«33 

(1-A 2 )*1 0 0 

0 (1-A 3 )A 2 0 


(1-Al)a„_i (\-h\)a n 

0 0 

0 0 


0 


0 


0 


(1 — 


0 


Thus, we see that {I — H)L is a matrix with the same mathematical form as a Leslie matrix. In Section 11.18 we showed that a 
necessary and sufficient condition for a Leslie matrix to have 1 as an eigenvalue is that its net reproduction rate also be 1 [see Eq. 
16 of Section 11.18]. Calculating the net reproduction rate of (/ _ H)L and setting it equal to 1, we obtain (verify) 

(1 — h\) [a\ + a 2 b\{\ — h 2 ) + ^ 1 ^ 2(1 — b 2 ){\ —^ 3 ) + "■ 

+ a»b l b 2 -b n - l 0-h 2 )(\ -A 3 )-(l -h n )] = 1 (4 


This equation places a restriction on the allowable harvesting fractions. Only those values of & ^ ..., h n that satisfy 4 and that 
lie in the interval [0, 1] can produce a sustainable yield. 


If Al> h 2 , •••> A„ do satisfy 4, then the matrix (1 — H)L has the desired eigenvalue Aj = 1- Furthermore, this eigenvalue has 
multiplicity 1, because the positive eigenvalue of a Leslie matrix always has multiplicity 1 (Theorem 11.18.1). This means that 
there is only one linearly independent eigenvector x satisfying Equation 2. [See Exercise 3(b) of Section 11.18.] One possible 
choice for x is the following normalized eigenvector: 


1 

W-h) 

b { b 2 (\-h 2 )(\-h 3 ) 

*i*a*3(l — *2)C1 — *3)C1 — *4) 


hb 2 b 3 -b n _ 1 (\-h 2 )(l-h 3 )-0-k n ) 

Any other solution x of 2 is a multiple of x 1 . Thus, the vector x j determines the proportion of females within each of the n classes 
after a harvest under a sustainable harvesting policy. But there is an ambiguity in the total number of females in the population 
after each harvest. This can be determined by some auxiliary condition, such as an ecological or economic constraint. For example, 
for a population economically supported by the harvester, the largest population the harvester can afford to raise between harvests 
would determine the particular constant that xj is multiplied by to produce the appropriate vector x in Equation 2. For a wild 
population, the natural habitat of the population would determine how large the total population could be between harvests. 


Summarizing our results so far, we see that there is a wide choice in the values of k j, h 2 -, •••, h n that will produce a sustainable 
yield. But once these values are selected, the proportional age distribution of the population after each harvest is uniquely 
determined by the normalized eigenvector _\i defined by Equation 5. We now consider a few particular harvesting strategies of this 
type. 


Uniform Harvesting 


With many populations it is difficult to distinguish or catch animals of specific ages. If animals are caught at random, we can 
reasonably assume that the same fraction of each age class is harvested. We therefore set 

h = h\ = h 2 = — = hyi 


Equation 2 then reduces to (verify) 



Hence, 1 / (1 — A) must be the unique positive eigenvalue Aj of the Leslie growth matrix L. That is. 


Solving for the harvesting fraction h, we obtain 


A = 1 — (1 / Ai) 


(6) 








The vector jq, in this case, is the same as the eigenvector of L corresponding to the eigenvalue h \ 1 . From Equation 8 of Section 
11.18, this is 

1 

*l/Ai 

*1*2/A? 

M_ b x b 2 b 2 i\\ (7: 

*l*2-*„_l/ A”” 1 

From 6 we can see that the larger / \ 1 is, the larger is the fraction of animals we can harvest without depleting the population. Note 
that we need Aj > 1 in order for the harvesting fraction h to lie in the interval (0, 1). This is to be expected, because Aj > 1 is the 
condition that the population be increasing. 


Harvesting Sheep 


For a certain species of domestic sheep in New Zealand with a growth period of 1 year, the following Leslie matrix was found (see 

G. Caughley, “Parameters for Seasonally Breeding Populations,” Ecology, 48, 1967, pp. 834-839). 

.000 .045 .391 .472 .484 .546 .543 .502 .468 .459 .433 .421 

.845 00000000000 

0 .975 0000000000 

0 0 .965 000000000 

0 0 0 .950 00000000 

, 0 0 0 0 .926 0 0 0 0 0 0 0 

L 0 0 0 0 0 .895 0 0 0 0 0 0 

0 0 0 0 0 0 .850 0 0 0 0 0 

0 0 0 0 0 0 0 .786 0 0 0 0 

00000000 .691 0 0 0 

00 0 000000 .561 0 0 

0000000000 .370 0 

The sheep have a lifespan of 12 years, so they are divided into 12 age classes of duration 1 year each. By the use of numerical 

techniques, the unique positive eigenvalue of L can be found to be 

Ai = 1.176 

From Equation 6, the harvesting fraction h is 

h = 1 - (1 / Ai) = 1 - (1 /1.176) = .150 

Thus, the uniform harvesting policy is one in which 15.0% of the sheep from each of the 12 age classes is harvested every year. 

From 7 the age distribution vector of the sheep after each harvest is proportional to 

1.000 
0.719 
0.596 
0.489 
0.395 
0.311 
0.237 
0.171 
0.114 
0.067 
0.032 
0.010 


xi = 


( 8 ) 










From 8 we see that for every 1000 sheep between 0 and 1 year of age that are not harvested, there are 719 sheep between 1 and 2 
years of age, 596 sheep between 2 and 3 years of age, and so forth. 

♦ 


Harvesting Only the Youngest Age Class 

In some populations only the youngest females are of any economic value, so the harvester seeks to harvest only the females from 
the youngest age class. Accordingly, let us set 

h\ = h 

h 2 = h 2 = ■■■ = h n = 0 

Equation 4 then reduces to 

(1 —h){a\ 4- a 2 b\ 4- a 2 b\b 2 4- ■" + a r fc\b?-b n -\) = 1 


or 

(\-k)R=\ 

where R is the net reproduction rate of the population. [See Equation 17 of Section 11.18.] Solving for h, we obtain 


h = \-(UK) 


(9) 


Note from this equation that a sustainable harvesting policy is possible only if R > ]. This is reasonable because only if £; > ] is the 
population increasing. From Equation 5, the age distribution vector after each harvest is proportional to the vector 


1 

^1 

b\b 2 

b\b 2 b 2 


( 10 ) 


bib 2 byb n _i 


Sustainable Harvesting Policy 


Let us apply this type of sustainable harvesting policy to the sheep population in Example 1. For the net reproduction rate of the 
population we find 

R = a\ -\- a 2 b\ +- a 2 b\b 2 4 - \-a yi b\b 2 “‘b yi -\ 

= (.000) 4- (.045)(.845) + -+ (.421)(.845)(.975)~(.370) 

= 2.514 

From Equation 9, the fraction of the first age class harvested is 

h = 1 - (1 !R) = 1 - (1 / 2.514) = .602 

From Equation 10, the age distribution of the sheep population after the harvest is proportional to the vector 






*1 = 


( 11 ) 




1.000 

0.845 

1.000 


0.824 

.845 


0.795 

(.845) (.975) 


0.755 

(.845) (.975) (.965) 


0.699 



0.626 

' 


0.532 

(.845) (.975)—(. 370) 


0.418 

0.289 



0.162 



0.060 


A direct calculation gives us the following (see also Exercise 3): 

2.514 
0.845 
0.824 
0.795 
0.755 
r 0.699 
M_ 0.626 
0.532 
0.418 
0.289 
0.162 
0.060 


( 12 ) 


The vector Lxi is the age distribution vector immediately before the harvest. The total of all entries in Lxi is 8.520, so the first 
entry 2.514 is 29.5% of the total. This means that immediately before each harvest, 29.5% of the population is in the youngest age 
class. Since 60.2% of this class is harvested, it follows that 17.8% (= 60.2% of 29.5%) of the entire sheep population is harvested 
each year. This can be compared with the uniform harvesting policy of Example 1, in which 15.0% of the sheep population is 
harvested each year. 

♦ 


Optimal Sustainable Yield 

We saw in Example 1 that a sustainable harvesting policy in which the same fraction of each age class is harvested produces a 
yield of 15.0% of the sheep population. In Example 2 we saw that if only the youngest age class is harvested, the resulting yield is 
17.8% of the population. There are many other possible sustainable harvesting policies, and each generally provides a different 
yield. It would be of interest to find a sustainable harvesting policy that produces the largest possible yield. Such a policy is called 
an optimal sustainable harvesting policy, and the resulting yield is called the optimal sustainable yield. However, determining the 
optimal sustainable yield requires linear programming theory, which we will not discuss here. We refer the reader to the following 
result, which appears in J. R. Beddington and D. B. Taylor, “Optimum Age Specific Harvesting of a Population,” Biometrics, 29, 
1973, pp. 801-809. 


THEOREM 11.19.1 


Optimal Sustainable Yield 

An optimal sustainable harvesting policy is one in which either one or two age classes are harvested. If two age classes are 
harvested, then the older age class is completely harvested. 


As an illustration, it can be shown that the optimal sustainable yield of the sheep population is attained when 









hi = 0.522 

kg = 1.000 


(13) 


and all other values of are zero. Thus, 52.2% of the sheep between 0 and 1 year of age and all the sheep between 8 and 9 years 
of age are harvested. As we ask the reader to show in Exercise 2, the resulting optimal sustainable yield is 19.9% of the population. 


Exercise Set 11.19 



Click here for Just Ask! 


1 . 


Let a certain animal population be divided into three 1-year age classes and have as its Leslie matrix 


L = 


0 

1 

2 

0 


4 3 
0 0 


(a) Lind the yield and the age distribution vector after each harvest if the same fraction of each of the three age classes is 
harvested every year. 

(b) Lind the yield and the age distribution vector after each harvest if only the youngest age class is harvested every year. 
Also, find the fraction of the youngest age class that is harvested. 


Lor the optimal sustainable harvesting policy described by Equations 13, find the vector xi that specifies the age distribution of 
2- the population after each harvest. Also calculate the vector £ Xl and verify that the optimal sustainable yield is 19.9% of the 
population. 

of an animal population is harvested, 


If only the 7th class of an animal population is to be periodically harvested (/ = 1, 2,find the corresponding harvesting 
4* fraction kj- 

Suppose that all of the /th class and a certain fraction hj of the 7th class of an animal population is to be periodically harvested 
5 * (!</</<«)• Calculate hj- 


3 . 


Use Equation 10 to show that if only the first age class 


Lx! - X! = 


R- 1 
0 
0 

0 


where 7? is the net reproduction rate of the population. 


Section 11.19 









Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


The results of Theorem 11.19.1 suggest the following algorithm for determining the optimal sustainable yield. 

Tl. 

(i) For each value of j = 1 , 2,..., n, set hj = h and h k = Q for h ±j and calculate the respective yields. These n 
calculations give the one-age-class results. Of course, any calculation leading to a value of h not between 0 and 1 is 
rejected. 

(ii) For each value of i = 1 , 2, ..., n — 1 and j = i 4 - 1, j + 2, ..., n, set hj — h, hj = 1, and h% = o for k * i, j and calculate 
the respective yields. These — 1) calculations give the two-age-class results. Of course, any calculation leading 
to a value of h not between 0 and 1 is again rejected. 

(iii) Of the yields calculated in parts (i) and (ii), the largest is the optimal sustainable yield. Note that there will be at most 

n + — 1 ) = -^«(« + 1 ) 

calculations in all. Once again, some of these may lead to a value of h not between 0 and 1 
and must therefore be rejected. 

If we use this algorithm for the sheep example in the text, there will be at most -1(12) (12 + 1) = 78 calculations to consider. 
Use a computer to do the two-age-class calculations for h\ = h, hj = 1, and h k = 0 f° r k^ \ or/ for j = 2, 3, ..., 12. Construct 
a summary table consisting of the values of h j and the percentage yields using j = 2,3, 12, which will show that the 

largest of these yields occurs when j = 9 . 

Using the algorithm in Exercise Tl, do the one-age-class calculations for hj = h and = 0 f° r k ^: f° r : = L 2, ..., 12. 

T2. Construct a summary table consisting of the values of hj and the percentage yields using j = 1,2, ..., 12, which will show that 
the largest of these yields occurs when j = 9 . 

Referring to the mouse population in Exercise T3 of Section 11.18, suppose that reducing the birthrates is not practical, so 
T3. you instead decide to control the population by uniformly harvesting all of the age classes monthly. 


(a) What fraction of the population must be harvested monthly to bring the mouse population to equilibrium eventually? 

(b) What is the equilibrium age distribution vector under this uniform harvesting policy? 

(c) The total number of mice in the original mouse population was 155. What would be the total number of mice after 5, 
10, and 200 months under your uniform harvesting policy? 
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11.20 

A LEAST SQUARES 
MODEL FOR HUMAN 
HEARING 


In this section we apply the method of least squares approximation to a model 
for human hearing. The use of this method is motivated by energy 
considerations. 


Inner Product Spaces 
Orthogonal Projection 
Fourier Series (Section 9.4) 


Anatomy of the Ear 

We begin with a brief discussion of the nature of sound and human hearing. Figure 11.20.1 is a schematic diagram of the ear 
showing its three main components: the outer ear, middle ear, and inner ear. Sound waves enter the outer ear where they are 
channeled to the eardrum, causing it to vibrate. Three tiny bones in the middle ear mechanically link the eardrum with the 
snail-shaped cochlea within the inner ear. These bones pass on the vibrations of the eardrum to a fluid within the cochlea. The 
cochlea contains thousands of minute hairs that oscillate with the fluid. Those near the entrance of the cochlea are stimulated by 
high frequencies, and those near the tip are stimulated by low frequencies. The movements of these hairs activate nerve cells that 
send signals along various neural pathways to the brain, where the signals are interpreted as sound. 



Figure 11.20.1 


The sound waves themselves are variations in time of the air pressure. For the auditory system, the most elementary type of sound 
wave is a sinusoidal variation in the air pressure. This type of sound wave stimulates the hairs within the cochlea in such a way that 
a nerve impulse along a single neural pathway is produced (Figure 11.20.2). A sinusoidal sound wave can be described by a 
function of time 


q{i)=A{\ Pj4sin(iitf — 6) ^ 

where q(t) is the atmospheric pressure at the eardrum, is the normal atmospheric pressure, A is the maximum deviation of the 
pressure from the normal atmospheric pressure, w / 2 tt is the frequency of the wave in cycles per second, and S is the phase angle of 
the wave. To be perceived as sound, such sinusoidal waves must have frequencies within a certain range. For humans this range is 
roughly 20 (cps) to 20,000 cps. Frequencies outside this range will not stimulate the hairs within the cochlea enough to produce 
nerve signals. 










Figure 11.20.2 


To a reasonable degree of accuracy, the ear is a linear system. This means that if a complex sound wave is a finite sum of 
sinusoidal components of different amplitudes, frequencies, and phase angles, say, 

q(t) = j4q 4 j4isin(a,'i£ — bq) 4 A 2 sm(^,' 2 £ ~$l) H-1- j4 M sin(u; H £ — 


then the response of the ear consists of nerve impulses along the same neural pathways that would be stimulated by the individual 
components (Figure 11.20.3). 




Figure 11.20.3 


Let us now consider some periodic sound wave p(t) with period T [i.e., p{t) =p{t \ T)\ that is not a finite sum of sinusoidal 
waves. If we examine the response of the ear to such a periodic wave, we find that it is the same as the response to some wave that 
is the sum of sinusoidal waves. That is, there is some sound wave q{£) as given by Equation 2 that produces the same response as 
p(t), even though p{{) and q(f ) are different functions of time. 

We now want to determine the frequencies, amplitudes, and phase angles of the sinusoidal components of q(t). Because q(t) 
produces the same response as the periodic wave p(t ), it is reasonable to expect that q(pj has the same period T as pit)- This 
requires that each sinusoidal term in q(t ) have period T. Consequently, the frequencies of the sinusoidal components must be 
integer multiples of the basic frequency 1 / 7 of the function pit)- Thus, the in Equation 2 must be of the form 

ajfc = 2£~/7’, £=1,2,... 

But because the ear cannot perceive sinusoidal waves with frequencies greater than 20,000 cps, we may omit those values of k for 
which Ljfc/ 27 r = £/ 7is greater than 20,000. Thus, q(f) is of the form 

■ 7 ( 0 = 4 ) 4 Aian^--6i ) j+- + A»aa(^--S„^ (3) 

where n is the largest integer such that« / 7 is not greater than 20,000. 

We now turn our attention to the values of the amplitudes j4j,. .., A n and the phase angles ^ • • •» S n that appear in Equation 
3. There is some criterion by which the auditory system “picks” these values so that q(t) produces the same response as p(t)- To 
examine this criterion, let us set 

e{t) =p(t) -0(0 

If we consider q(f) as an approximation to p(t), then s (i) is the error in this approximation, an error that the ear cannot perceive. 









































In terms of e(t), the criterion for the determination of the amplitudes and the phase angles is that the quantity 



(4) 


o 7o 


be as small as possible. We cannot go into the physiological reasons for this, but we note that this expression is proportional to the 
acoustic energy of the error wave e(t") over one period. In other words, it is the energy of the difference between the two sound 
waves p( f ) and q(f) that determines whether the ear perceives any difference between them. If this energy is as small as possible, 
then the two waves produce the same sensation of sound. Mathematically, the function q(t ) in 4 is the least squares approximation 
to p(t ) from the vector space C[0, T] of continuous functions on the interval [0, T]. (See Section 9.4.) 

Least squares approximations by continuous functions arise in a wide variety of engineering and scientific approximation 
problems. Apart from the acoustics problem just discussed, some other examples follow. 


Let S(x) be the axial strain distribution in a uniform rod lying along the v-axis from x = 0 to x = / (Figure 11.20.4). The 
strain energy in the rod is proportional to the integral 



The closeness of an approximation q { x ) to S{x) can be judged according to the strain energy of the 
difference of the two strain distributions. That energy is proportional to 



which is a least squares criterion. 







Figure 11.20.4 


2. Let E{i) be a periodic voltage across a resistor in an electrical circuit (Figure 1 1.20.5). The electrical energy transferred to 
the resistor during one period T is proportional to 



If q{t) has the same period as E{t) and is to be an approximation to E{t), then the criterion of 
closeness might be taken as the energy of the difference voltage. This is proportional to 



which is again a least squares criterion. 


ji E{i) 

voltage 










Figure 11.20.5 


Let y (x ) be the vertical displacement of a uniform flexible string whose equilibrium position is along the v-axis from = 0 
to x = i (Figure 11.20.6). The elastic potential energy of the string is proportional to 



2 

bOO] dx 


If q( x ) is to be an approximation to the displacement, then as before, the energy integral 



[y(x)-q(x)] 2 dx 


determines a least squares criterion for the closeness of the approximation. 



Least squares approximation is also used in situations where there is no a priori justification for its use, such as for approximating 
business cycles, population growth curves, sales curves, and so forth. It is used in these cases because of its mathematical 
simplicity. In general, if no other error criterion is immediately apparent for an approximation problem, the least squares criterion 
is the one most often chosen. 

The following result was obtained in Section 9.4. 


THEOREM 11.20.1 


Minimizing Mean Square Error on [0 f 2jt ] 


If /(£) is continuous on [0, 2tt], then the trigonometric function g{j^ of the form 

g(t) = -i-ag f a\ cos t + — 4 cos nt + b\ sin£ 4-h b n sin nt 

that minimizes the mean square error 

f 2 ~ o 


/ U{t)-g{t)] 2 dt 

J 0 

has coefficients 

1 / 

n 2jr 

a k = ~ I 

/ (£) cos k£d£, k = 0, 1, 2, n 

) 


"2tt 

bk = ± 
*h 

/ (£) sin k£ d£, 

) 


If the original function j (£) is defined over the interval [0, T\ instead of f 0, 2 tt 1 , a change of scale will yield the following result 
(see Exercise 8): 





THEOREM 11.20.2 


Minimizing Mean Square Error on [0, T] 

If f (t) is continuous on [0, T], then the trigonometric function g(f) of the form 

g{t) = i- a 1 cos ^rt + - + a„ cos+ b i sin^ + - + b n sin ^-t 


that minimizes the mean square error 


f\ 

Jo 


has coefficients 


-y 

bk= T.I 0 




2 kKt 


f{t)zos^dt, * = 0,1,2.n 

/(Osin ^-di. *= 1 , 2 ,..., n 


Least Squares Approximation to a Sound Wave 


Let a sound wave p(t ) have a saw-tooth pattern with a basic frequency of 5000 cps (Figure 11.20.7). Assume units are chosen so 
that the normal atmospheric pressure is at the zero level and the maximum amplitude of the wave is A. The basic period of the 
wave is T= 1 / 5000 = .0002 second. From t = 0 to t = 7, the function p(t) has the equation 



Theorem 11.20.2 then yields the following (verify): 



*= 1 , 2 ,... 

*= 1 , 2 ,... 


We can now investigate how the sound wave p(t) is perceived by the human ear. We note that 4/7= 20,000 cps, so we need only 
go up to l : = 4 in the formulas above. The least squares approximation to p(f) is then 




sin^f 4 4 3™^ 4 4 sin ^ 


The four sinusoidal terms have frequencies of 5000, 10,000, 15,000, and 20,000 cps, respectively. In Figure 11.20.8 we have 
plotted p(f) and q(t) over one period. Although q(t) is not a very good point-by-point approximation to p(f), to the ear, both p(t) 
and q(t) produce the same sensation of sound. 

♦ 




















Figure 11.20.7 



As discussed in Section 9.4, the least squares approximation becomes better as the number of terms in the approximating 
trigonometric polynomial becomes larger. More precisely, 


/ |"/(O-i«0- H Ofccos kt I b^sinkt) 

Jo I 1 fc=l 



tends to zero as n approaches infinity. We denote this by writing 

I oo 

f + 5Z (a ft cos to -I- bfr sin to) 

4 fc=l 


where the right side of this equation is the Fourier series of f (t). Whether the Fourier series of f (i) converges to f (t) for each t 
is another question, and a more difficult one. For most continuous functions encountered in applications, the Fourier series does 
indeed converge to its corresponding function for each value of t. 


Exercise Set 11 .20 



Click here for Just Ask! 


-i 

Find the trigonometric polynomial of order 3 that is the least squares approximation to the function f (t) = (t — x) over the 
interval [0, 2x]. 

Find the trigonometric polynomial of order 4 that is the least squares approximation to the function f (t) =t? over the interval 
2 ' [0.7]. 


Find the trigonometric polynomial of order 4 that is the least squares approximation to the function f (() over the interval 
3* [0,2~], where 


/CO 


sin£, 0 <£<tt 
0, 7T <. t 2.X 


Find the trigonometric polynomial of arbitrary order n that is the least squares approximation to the function / (t) = sin-jU over 

4 . 4 

the interval [0, 2x]. 


Find the trigonometric polynomial of arbitrary order n that is the least squares approximation to the function f (t) over the 







interval [0, T\, where 


t. 


/(0 = 


T-t, 


0<t<±T 
±T<t<T 


6 . 


For the inner product 


/■ 2 ~ 

(u, v) = / u 
' 1 JO 


(t)v(t)dt 


show that 


(a) || 11) = 


(b) ||cosfe|| = ^7r for£=l,2. 


(c) ||sinfe|| = for£=l,2. 


Show that the 2n h 1 functions 

7 

1, cos£, cos 2 1 ,..., cosnt, sin^, sin2£, iva.nl 

are orthogonal over the interval [0, 2k] relative to the inner product (u, vj defined in Exercise 6. 

If f (t) is defined and continuous on the interval [0, T], show that j (Tr / 2~") is defined and continuous for T in the interval 
8* [ 0, 2~]. Use this fact to show how Theorem 11.20.2 follows from Theorem 11.20.1. 


Section 11.20 



Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Tl. 


Let g be the function 


g(0 = 


3 I 4 sin t 

5—4 cos£ 


for 0 < t < 2n- Use a computer to determine the Fourier coefficients 

f fl *l _ 1 f 27r ( 3 I 4 sin l 
pjcJ kJo \ 5 “4 cos t 


COS fe 

sinfe 


di 


for fc — 0, 1, 2, 3, 4, 5. From your results, make a conjecture about the general expressions for and Test your conject 
by calculating 

1 00 

o+ X! (flfccosfe 4- b k sinfe) 

2 k=l 






on the computer and see whether it converges to g(£). 


T2. 


Let g be the function 


g(t) = e cos f [cos(sm^) I sin(sin^)] 
for 0 < t < 2tt- Use a computer to determine the Fourier coefficients 



for k = 0, 1, 2, 3, 4, 5. From your results, make a conjecture about the general expressions for a% and h k: Test your conjecture 
by calculating 

1 1X1 

■^0+ 21 (flfccosfe f &fcsin&) 

2 h =1 


on the computer and see whether it converges to g(^). 
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11.21 

WARPS AND MORPHS 


Among the more interesting image-manipulation techniques available for 
computer graphics are warps and morphs. In this section we show how linear 
transformations can be used to distort a single picture to produce a warp, or to 
distort and blend two pictures to produce a morph. 


Geometry of Linear Operators on p} (Section 9.2) 
Linear Independence 
Bases in p 2 


Most computer graphics software enables you to manipulate an image in various ways, such as by scaling, rotating, or slanting the 
image. Distorting an image by moving the corners of a rectangle containing the image is another basic image-manipulation 
technique. Distorting various pieces of an image in different ways is a more complicated procedure that results in a warp of the 
picture. In addition, warping two different images in complementary ways and blending the warps results in a morph of the two 
pictures (from the Greek root meaning “shape” or “form”). The main application of warping and morphing images has been the 
production of special effects in motion pictures and television or print advertisements. However, many scientific and technological 
applications for such techniques have also arisen—for example, studying the evolution of the shapes of living organisms, 
analyzing the growth and development of living organisms, assisting in reconstructive and cosmetic surgery, investigating 
variations in the design of a product, and “aging” photographs of missing people or police suspects. 


Warps 

We begin by describing a simple warp of a triangular region in the plane. Let the three vertices of a triangle be given by the three 
noncollinear points \q, V2, and v 3 (Figure 11.21.1a). We shall call this triangle the begin-triangle . If v is any point in the 
begin-triangle, then there are unique constants cj and C 2 such that 

V - v 3 = c 1 (vi - v 3 ) -I- c 2 (v2 - V 3 ) Q 

Equation 1 expresses the vector v — v 3 as a (unique) linear combination of the two linearly independent vectors \q — v 3 and 
V2 — v 3 with respect to an origin at v 3 . If we set = \ _ c\ — C2> then we can rewrite 1 as 

V = ClVI -I- C2V 2 -I- c 3 v 3 


where 


ci +c 2 fc 3 = 1 ^ 

from the definition of c 3 . We say that v is a convex combination of the vectors vq, v 2 , and v 3 if 2 and 3 are satisfied and, in 
addition, the coefficients c\, c 2 , and c 3 are nonnegative. It can be shown (Exercise 6) that v lies in the triangle determined by vp v 2 
, and v 3 if and only if it is a convex combination of those three vectors. 




(tf) 



« = CjW, +■ I'jHS + 

(A) 

Figure 11.21.1 

Next, given three noncollinear points wi, w 2 , and w 3 of an end-triangle (Figure 11.21.1 b), there is a unique affine transformation 
that maps vi to wq, v 2 to w 2 , and v 3 to w 3 . That is, there is a unique 2x2 invertible matrix M and a unique vector b such that 

w, = M\i + h fori = 1,2, 3 ^ 

(See Exercise 5 for the evaluation of M and b .) Moreover, it can be shown (Exercise 3) that the image w of the vector v in 2 under 
this affine transformation is 


W = C1W1 + C 2 W 2 +C 3 W 3 

This is a basic property of affine transformations: They map a convex combination of vectors to the same convex combination of 
the images of the vectors. 

Now suppose that the begin-triangle contains a picture within it (Figure 11.21.2 a). That is, to each point in the begin-triangle we 
assign a gray level, say 0 for white and 100 for black, with any other gray level lying between 0 and 100. In particular, let a 
scalar-valued function p$, called the picture-density of the begin-triangle, be defined so that ■,q ( v ) is the gray level at the point v in 
the begin-triangle. We can now define a picture in the end-triangle, called a warp of the original picture, with a picture-density p\ 
by defining the gray level at the point w within the end-triangle to be the gray level of the point v in the begin-triangle that maps 
onto w. In equation form, the picture-density p\ is determined by 

pl(w) =Po(civi f C 2 v 2 + c 3 v 3 ) 

In this way, as cj, c 2 , and c 3 vary over all nonnegative values that add to one, 5 generates all points w in the end-triangle, and 6 
generates the gray levels ( w ) of the warped picture at those points (Figure 11.21.2 b). 








ih) 

Figure 11.21.2 


Equation 6 determines a very simple warp of a picture within a single triangle. More generally, we can break up a picture into 
many triangular regions and warp each triangular region differently. This gives us much freedom in designing a warp through our 
choice of triangular regions and how we change them. To this end, suppose we are given a picture contained within some 
rectangular region of the plane. We choose n points vj, \' 2 , ..., v M within the rectangle, which we call vertex points, so that they 
fall on key elements or features of the picture we wish to warp (Figure 11.21.3a). Once the vertex points are chosen, we complete 
a triangulation of the rectangular region; that is, we draw lines between the vertex points in such a way that we have the following 
conditions (Figure 11.21.3Z?): 


The lines form the sides of a set of triangles. 


2. The lines do not intersect. 


3 Each vertex point is the vertex of at least one triangle. 


4, The union of the triangles is the rectangle. 


5, The set of triangles is maximal (that is, no more vertices can be connected). 


Note that condition 4 requires that each comer of the rectangle containing the picture be a vertex point. 







v l V, 



k) 

Figure 11.21.3 


One can always form a triangulation from any n vertex points, but the triangulation is not necessarily unique. For example, Figures 
11.21.3Z? and 11.21.3c are two different triangulations of the set of vertex points in Figure 11.21.3a. Since there are various 
computer algorithms that perform triangulations very quickly, it is not necessary to perform the tiresome triangulation task by 
hand; one need only specify the desired vertex points and let a computer generate a triangulation from them. If n is the number of 
vertex points chosen, it can be shown that the number of triangles m of any triangulation of those points is given by 

m = 2n^2~k ^ 

where k is the number of vertex points lying on the boundary of the rectangle, including the four situated at the corner points. 

The warp is specified by moving the n vertex points \q, v?, v H to new locations wq, W 2 , w H according to the changes we 
desire in the picture (Figures 11.21.4a and 11.21.4Z?). However, we impose two restrictions on the movements of the vertex points: 


The four vertex points at the corners of the rectangle are to remain fixed, and any vertex point on a side of the rectangle is to 
remain fixed or move to another point on the same side of the rectangle. All other vertex points are to remain in the interior 
of the rectangle. 

2. The triangles determined by the triangulation are not to overlap after their vertices have been moved. 

The first restriction guarantees that the rectangular shape of the begin-picture is preserved. The second restriction guarantees that 
the displaced vertex points still form a triangulation of the rectangle and that the new triangulation is similar to the original one. 

For example. Figure 11.21.4c is not an allowable movement of the vertex points shown in Figure 11.21.4a. Although a violation of 






this condition can be handled mathematically without too much additional effort, the resulting warps usually produce unnatural 
results and we shall not consider them here. 


V| V, W, IV, VV, 




Figure 11.21.4 


Figure 11.21.5 is a warp of a photograph of a woman using a triangulation with 94 vertex points and 179 triangles. Note that the 
vertex points in the begin-triangulation are chosen to lie along key features of the picture (hairline, eyes, lips, etc.). These vertex 
points were moved to final positions corresponding to those same features in a picture of the woman taken 20 years after the 
begin-picture. Thus, the warped picture represents the woman forced into her older shape but using her younger gray levels. 




Begin-lriungii latiim 



Wuipcd iriyngukitmn 



Lkgiit-lriangulfititfn 



Warptd kimigulaikin 


Figure 11.21.5 


































Time-Varying Warps 


A time-varying warp is the set of warps generated when the vertex points of the begin-picture are moved continually in time from 
their original positions to specified final positions. This gives us a motion picture in which the begin-picture is continually warped 
to a final warp. Let us choose time units so that t = o corresponds to our begin-picture and t = \ corresponds to our final warp. The 
simplest way of moving the vertex points from time 0 to time 1 is with constant velocity along straight-line paths from their initial 
positions to their final positions. 


To describe such a motion, let «,•(£) denote the position of the /th vertex point at any time t between 0 and 1. Thus u, (0) = v,- (its 
given position in the begin-picture) and u, (1) = w, (its given position in the final warp). In between, we determine its position by 

= 0 -Ovj-Mw, 


Note that 8 expresses u, (£) as a convex combination of Vj and w, for each t in [0, 1]. Figure 11.21.6 illustrates a time-varying 
triangulation of a plain rectangular region with six vertex points. The lines connecting the vertex points at the different times are 
the space-time paths of these vertex points in this space-time diagram. 



Once the positions of the vertex points are computed at time t, a warp is performed between the begin-picture and the triangulation 
at time t determined by the displaced vertex points at that time. Figure 11.21.7 shows a time-varying warp at five values of t 
generated from the warp between t = 0 and t = \ shown in Figure 11.21.5. 



I = 0.00 / = 0.25 I =0.50 I =0.25 / = 1.00 


Figure 11.21.7 


Morphs 

A time-varying morph can be described as a blending of two time-varying warps of two different pictures using two triangulations 
that match corresponding features in the two pictures. One of the two pictures is designated as the begin-picture and the other as 
the end-picture. First, a time-varying warp from t = o to t = \ is generated in which the begin-picture is warped into the shape of 
the end-picture. Then a time-varying warp from t = \ to t = o is generated in which the end-picture is warped into the shape of the 















begin-picture. Finally, a weighted average of the gray levels of the two warps at each time t is produced to generate the morph of 
the two images at time t. 

Figure 11.21.8 shows two photographs of a woman taken 20 years apart. Below the pictures are two corresponding triangulations 
in which corresponding features of the two photographs are matched. The time-varying morph between these two pictures for five 
values of t between 0 and 1 is shown in Figure 11.21.9. 


Begin-picture 




End-picture 




Begin -1 rki rtgu I ill km l E nd -1 riuiiguklE ion 

Figure 11.21.8 



f = (1.00 t = 0.25 i = 0.75 /=].00 

Figure 11.21.9 


The procedure for producing such a morph is outlined in the following nine steps (Figure 11.21.10): 

Step 1. Given a begin-picture with picture-density pg and an end-picture with picture-density p\, position n vertex points vi, 
V 2 , ..., v H in the begin-picture at key features of that picture. 















Figure 11.21.10 


Tunc - I 

End-picture 

Given density: ft ( (w ] 


Time = i 

Morph-picture 

Complied den*hy: 

p t W = G - i}Pi0)+ fpi(w) 


Time = 0 
Bc^m-ptctiue 
(liven density: \ 


Step 2. Position n corresponding vertex points wi, W 2 , ..w H in the end-picture at the corresponding key features of that 
picture. 

Step 3. Triangulate the begin- and end-pictures in similar ways by drawing lines between corresponding vertex points in both 
pictures. 


Step 4. For any time t between 0 and 1, find the vertex points Ul (£), 113 (O’ • • •> u„(£) in the morph picture at that time, using 
the formula 


Uj (£) = (1 — i)vj 4- tviu i = 1, 2, n 


(9) 


Step 5. Triangulate the morph picture at time t similar to the begin- and end-picture triangulations. 

Step 6. For any point u in the morph picture at time t, find the triangle in the triangulation of the morph picture in which it lies 
and the vertices u^), and of that triangle. (See Exercise 1 to determine whether a given point lies in a given 

triangle.) 


Step 7. Express u as a convex combination of uj(£)> u _/(£)> and ug-(t) by finding the constants cj, cj, and eg such that 


u = c/uj(0 +cjnj(t) + cjsriijfCO 


( 10 ) 


and 


CJ + CJ + C£= 1 


(ID 


Step 8. Determine the locations of the point u in the begin- and end-pictures using 


v = c jvj F c jvj I- egyz (in the begb-picture) 


(12) 













and 


w = c;w/ 4- c jwj 4- cgwg; (in the end-picture) 


(13) 


Step 9. Finally, determine the picture-density p t (ii) of the morph-picture at the point u using 


Mu) = (1 - i)po(v) -I- tpx (w) 

Step 9 is the key step in distinguishing a warp from a morph. Equation 14 takes weighted averages of the gray levels of the begin- 
and end-pictures to produce the gray levels of the morph-picture. The weights depend on the fraction of the distances that the 
vertex points have moved from their beginning positions to their ending positions. For example, if the vertex points have moved 
one-fourth of the way to their destinations (that is, if t = 0.25). then we use one-fourth of the gray levels of the end-picture and 
three-fourths of the gray levels of the begin-picture. Thus, as time progresses, not only does the shape of the begin-picture 
gradually change into the shape of the end-picture (as in a warp) but the gray levels of the begin-picture also gradually change into 
the gray levels of the end-picture. 

The procedure described above to generate a morph is cumbersome to perform by hand, but it is the kind of dull, repetitive 
procedure at which computers excel. A successful morph demands good preparation and requires more artistic ability than 
mathematical ability. (The software designer is required to have the mathematical ability.) The two photographs to be morphed 
should be carefully chosen so that they have matching features, and the vertex points in the two photographs also should be 
carefully chosen so that the triangles in the two resulting triangulations contain similar features of the two pictures. When the 
procedure is done correctly, each frame of the morph should look just as “real” as the begin- and end-pictures. 

The techniques we have discussed in this section can be generalized in numerous ways to produce much more elaborate warps and 
morphs. For example: 


If the pictures are in color, the three components of the picture colors (red, green, and blue) can be morphed separately to 
produce a color morph. 

2. Rather than following straight-line paths to their destinations, the vertices of a triangulation can be directed separately along 
more complicated paths to produce a variety of results. 

3. Rather than travel with constant speeds along their paths, the vertices of a triangulation can be directed to have different 
speeds at different times. For example, in a morph between two faces, the hairline can be made to change first, then the 
nose, and so forth. 


Similarly, the gray-level mixing of the begin-picture and end-picture at different times and different vertices can be varied in 
a more complicated way than that in Equation 14. 

5. One can morph two surfaces in three-dimensional space (representing two complete heads, for example) by triangulating the 
surfaces and using the techniques in this section. 

6 One can morph two solids in three-dimensional space (for example, two three-dimensional tomographs of a beating human 
heart at two different times) by dividing the two solids into corresponding tetrahedral regions. 

Two film strips can be morphed frame by frame by different amounts between each pair of frames to produce a morphed 
film strip in which, say, an actor walking along a set is gradually morphed into an ape walking along the set. 


8, Instead of using straight lines to triangulate two pictures to be morphed, more complicated curves, such as spline curves, can 
be matched between the two pictures. 


9 Three or more pictures can be morphed together by generalizing the formulas given in this section. 

These and other generalizations have made warping and morphing two of the most active areas in computer graphics. 


Exercise Set 11.21 


Click here for Just Ask! 


Determine whether the vector v is a convex combination of the vectors vi, V 2 , and Y 3 . Do this by solving Equations 1 and 3 for 
1 * ci, C 2 , and C 3 and ascertaining whether these coefficients are nonnegative. 


(a) 


v = 


"3" 


V 


'3' 


~A~ 

_3_ 

»vi = 

_ 1 _ 

> V2 = 

_5_ 

» v 3 = 

2 _ 


(b) 


v = 


" 2 " 


V 


'3' 


'A' 

_4_ 

»vi = 

_ 1 _ 

’ v 2 = 

_5_ 

» v 3 = 

2 _ 


(C) 


V = 


'o' 


r 31 


' - 2 ' 


r 31 

1 - 

O 

< 

II 

1 

_1 

< 

to 

ll 

_ -2_ 

•▼3 = 

1 

0 

_1 


(d) 


v = 


'r 


'3' 


' -2' 


'3' 

_ 0 _ 

* vi = 

_3_ 

’ V2 = 

_ - 2_ 

’ v 3 = 

_ 0 _ 


2 . 


Verify Equation 7 for the two triangulations given in Figure 11.21.3. 


Let an affine transformation be given by a 2 x 2 matrix M and a two-dimensional vector b. Let v = civj 4 C2V2 4- C3V3, where 
3- ci + C2 -I- C3 = 1; let w = Mv 4 - b; and let w - = M\ i 4 - b for i = 1 , 2, 3. Show that w = cjwi 4- C2W2 4- C3W3. (This shows that 
an affine transformation maps a convex combination of vectors to the same convex combination of the images of the vectors.) 


4. 


(a) Exhibit a triangulation of the points in Figure 11.21.3 in which the points V 3 , vy and vg form the vertices of a single 
triangle. 


(b) Exhibit a triangulation of the points in Figure 11.21.3 in which the points V 2 , vy and V 7 do not form the vertices of a 
single triangle. 


5. 


Find the 2x2 matrix M and two-dimensional vector b that define the affine transformation that maps the three vectors vy \'2, 
and V 3 to the three vectors wy W 2 > and W 3 . Do this by setting up a system of six linear equations for the four entries of the 



































matrix M and the two entries of the vector b. 



6 . 

(a) Let a and b be linearly independent vectors in the plane. Show that if c\ and C2 are nonnegative numbers such that 
c i | C2 = 1* then the vector C \<i + C2b lies on the line segment connecting the tips of the vectors a and b. 

(b) Let a and b be linearly independent vectors in the plane. Show that if c\ and C2 are nonnegative numbers such that 
c\ 4 C2 < 1’ then the vector c j ^ _|_ ^b h es in the triangle connecting the origin and the tips of the vectors a and b. 

Hint First examine the vector C \<\ 4- C2b multiplied by the scale factor \! {c\ -\- C2)- 

(c) Let vi, V2, and V3 be noncollinear points in the plane. Show that if c\ r $2, and cj are nonnegative numbers such that 
C 1 + c 2 + c 3 = h then the vector cjvq 4 C2 V 2 + C 3 V 3 hes in the triangle connecting the tips of the three vectors. 

Hint Let a = vj — V3 and 1, = V2 — y 3 , and then use Equation 1 and part (b) of this exercise. 


7. 

(a) What can you say about the coefficients c\,c 2, and C3 that determine a convex combination v = cp'i + C2V2 4 C3V3 if v 
lies on one of the three vertices of the triangle determined by the three vectors vq, V2» and V3? 

(b) What can you say about the coefficients c\,C2, and C 3 that determine a convex combination v = civi 4- C2 V 2 + C 3 V 3 if v 
lies on one of the three sides of the triangle determined by the three vectors vq, \ r 2, and V3? 

(c) What can you say about the coefficients c\,c 2, and C3 that deter mi ne a convex combination v = cpq 4 - C2 V 2 d C 3 V 3 if v 
lies in the interior of the triangle determined by the three vectors \q, V2» and V3? 


8 . 

(a) The centroid of a triangle lies on the line segment connecting any one of the three vertices of the triangle with the 

midpoint of the opposite side. Its location on this line segment is two-thirds of the distance from the vertex. If the three 
vertices are given by the vectors vp V2, and V3, write the centroid as a convex combination of these three vectors. 



and 


Use your result in part (a) to find the vector defining the centroid of the triangle with the three vertices 

V 

1 ' 
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Technology Exercises 


The following exercises are designed to be solved using a technology utility. Typically, this will be matlab, Mathematica, Maple, 
Derive, or Mathcad, but it may also be some other type of linear algebra software or a scientific calculator with some linear algebra 
capabilities. For each exercise you will need to read the relevant documentation for the particular utility you are using. The goal of 
these exercises is to provide you with a basic proficiency with your technology utility. Once you have mastered the techniques in 
these exercises, you will be able to use your technology utility to solve many of the problems in the regular exercise sets. 


Tl. 



vii 


V 21 


V31 

To warp or morph a surface in ^3 we must be able to triangulate the surface. Let vj = 

1 

C-q m 

> > 
_1 

’ v 2 = 

y 22 

y 23 

, and v 3 = 

-1 

0-4 on 
on on 
> > 
_ 1 


three noncollinear vectors on the surface. Then a vector v = 


v 2 


v 3 


lies in the triangle formed by these three vectors if and oi 


if v is a convex combination of the three vectors; that is, v = cjvi -I- cv y 2 + "3 V 3 for some nonnegative coefficients c\, c 2, * 
C2 whose sum is 1. 


(a) Show that in this case, c\,c 2, and C3 are solutions of the following linear system: 


" y n 

V 21 

V 31~ 



‘vf 

v 12 

v 22 

y 32 

_ 

v 2 

c 2 

V13 

v 23 

V33 


v 3 

1 

1 

1 

c 3 


1 




In parts (b)-(d) determine whether the vector v is a convex combination of the vectors 



2 


'3' 


2 

vi = 

7 

-5 

t y 2 = 

0 

9 

, and v 3 = 

1_ 


(b) 


v 


1 

4 


9 

9 

9 


(c) 


v 


1 

4 


10 

9 

9 


































Vll 

T2. To warp or morph a solid object in p}' we first partition the object into disjoint tetrahedrons. Let vj = v\2 , v? = 

v 13 _ 

v 3 i i rv 4 ii rvr 

v r 3 = V32 , and V4 = V42 be four noncoplanar vectors. Then a vector v = V2 lies in the solid tetrahedron formed by these 

V33 V43 V3 

four vectors if and only if v is a convex combination of the three vectors; that is, v = cjvi f c-pyj I C3V3 -| C4V4 for some 
nonnegative coefficients cj, C2> ^3. and c4 whose sum is one. 

(a) Show that in this case, cj, C2, cj, and c 4 are solutions of the following linear system: 

vii v 2 i V31 v 4 i 

v 12 ^22 V32 V42 
v 13 v 2 3 V33 V43 
1111 

In parts (b)-(d) determine whether the vector v is a convex combination of the vectors 

r 2 1 [-3] [7] [-1 

vj = — 6 / \'2 = 4 , V 3 = 2 , and V 4 = 3 

1 J 2 J [3J |_2 

(b) 5 
v= 0 


(c) 1 

v= 1 


(d) 1 

v — 2 
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Abbreviations 


ARTs algebraic reconstruction techniques CE civil engineer cps cycles per second EE electrical engineer ME mechanical 
engineer RDA recommended daily allowance 
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Answers to Exercises 


Exercise Set 1.1 (page 6) 

1. (a), (c), (f) 


3. 


4. 


5. 


(a) 






(b) 

5 4 

*i = 3*“ r + 

l -4 


*2 = s 

*2=* 


X3 =t 

X3 = s 



X4 = £ 



'3 

— 

2 

— 

r 




(a) 

4 


5 


3 





7 


3 


2 





'2 


0 


2 

f 



(b) 

3 

— 

1 


4 

7 




6 


1 

— 

1 

0 




'1 

2 

0 

— 

1 


1 

1 

(c) 

0 

3 

1 


0 


• 1 

2 


0 

0 

1 


7 


0 
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'1 

0 

0 

r 





(d) 

0 

1 

0 

2 






0 

0 

1 

3 






2 x \ =0 

(a) 3xi -4x2 = 0 

*2 = 1 

3xj —2^3 

(b) 7xi +*2 + 4*3 

- 2 x 2 + *3 
7 xi f- 2x2 + *3 
^ xi + 2x2 + 4x3 


= 5 

= -3 
= 7 

-3x4 = 5 
= 1 


*4 

w —q 


x = r 


y=s 
z = t 











6 . 


(d) 


*1 


x 2 


x 3 = 


x 4 = 


1 

-2 

3 

4 


*- 2 7 = 5 
(a) 


Let x —1 \ then t- 2 y = 5 - Solving for y yields y = \t- 
(b) 4 4 


The lines have no common point of intersection. 

(a) 

The lines intersect in exactly one point. 

(b) 

The three lines coincide. 

(c) 


Exercise Set 1.2 (page 19) 

1. (a), (b), (c), (d), (h), (i), (j) 


Both 

(a) 


Neither 

(b) 


Both 

(c) 


Row-echelon 

(d) 


Neither 

(e) 


Both 

(f) 


xi = — 3 > xi = 0 > X3 = 7 
(a) 1 


(b) 


-t- 


x\=lt-[ 8^X2= — 3 t 4 - 2 > X3 = 


x 4 = t 


*1 = 6 s — 2 >t — 2 , *2 — s ’ *3 = - 4 t-\-l,x^ = - 5 t-\-^,x^ = t 

(c) 

Inconsistent 

(d) 


*1 = 3 , X 2 = 1 > ^3 = 2 

(a) 


13 14 

(b) *l=-i-?-*2 = j~ T *3=‘ 


x — t — y = 2 s, z — s, w = t 

(c) 


Inconsistent 

(d) 


Inconsistent 

(a) 


x\ = — A, xi = 2 , *3 = 7 
(b) 


xi = 3 4- 2 t, xi — t 

(c) 

(d) X =5~5 £ ~5 S,y =W ' f^“lV £,? = ^ w = s 


12 . (a), (c), (d) 


*1 = 0? X2 = 0> JT3 = 0 

(a) 


x 1 = —S,x 2 = — t — S, X? = As, XA = t 

(b) 


w=ia= — t, y = t, z = 0 
(c) 


Only the trivial solution 
(a) 


u = 7 s — 5 t, v = — 6 s At, y? = 2 s, x = 2 1 
(b) 


Only the trivial solution 
(c) 


15. 


/ 1= -l,/ 2 = 0./3 = l./ 4 = 2 


(a) 


(b) 


Zi = — s — U Zj = Z3 = — Z4 = O’ Z5 = £ 


19. 


1 3 
0 1 


and 


1 0 
0 1 


are possible answers. 


Q = W 2,0 = ir,7 = O 


23. 


If A = 1 , then X{=X2 = _ = s 

If A = 2 , then X{= _ l s , X2 = 0 , *3 = s 


24. x= - 13 / 7 , y = 91 / 54 , z = - 91/8 


a = 1,A= - 6,c = 2,<* = 10 


Three lines, at least two of which are distinct 
(a) 


Three identical lines 


(b) 


32. 


False 


(a) 


False 


(b) 


False 


(c) 


False 


(d) 


Exercise Set 1.3 (page 34) 


Undefined 

(a) 






4x2 


(b) 


Undefined 

(c) 

Undefined 

(d) 


5x5 

(e) 

5x2 

(f) 


Undefined 

(g) 

5x2 

(h) 


a = 5,b= -3,c = 4,rf = 1 


4. 


(a) 


7 2 4 
3 5 7 


(b) 


(c) 


-5 0-1 

4-1 1 

- 1-1 1 

-5 0-1 

4-1 1 

- 1-1 1 


Undefined 

(d) 


(e) 


3 
2 

4 0 

£ 

4 


(f) 


0 

1 


1 

0 


(g) 


9 1-1 
-13 2 -4 
0 1-6 


(h) 


9-13 0 

1 2 1 
-1 -4 -6 
















5. 



12 


■3 


(a) 

-4 


5 



4 


1 


Undefined 


(b) 






'42 

108 75' 

(c) 

12 

-3 21 


36 

78 63 


' 3 

45 

9 

(d) 

11 

-11 

17 


7 

17 

13 


' 3 

45 

9 

(e) 

11 

-11 

17 


7 

17 

13 


'21 

17' 



(f) 

17 

35_ 




' 0 

-2 11' 

(g) 

12 

1 

8 _ 


'12 


6 

9 

(h) 

48 

-20 

14 


24 


8 

16 


61 

(i) 

35 

(j) 

( 28 ) 

(k) 


8. 

67 


3 


-2 


7 

(a) 

64 

= 6 

6 

+ 0 

5 

+ 7 

4 


63 


0 


4 


9 


41' 


'3' 


' -2' 


21 

= -2 

6 

+ 1 

5 

+ 7 

67 


0 


4 



41 


3 


-2 


7 

59 

— 4 

6 

+ 3 

5 

+ 5 

4 

57 


0 


4 


9 












































(b) 


6 


6 


-2 


4 

6 

= 3 

0 

+ 6 

1 

+ 0 

3 

63 


7 


7 


5 


' -6' 
17 

= -2 

'6' 

0 

+ 5 

" -2' 
1 

+ 4 

"4" 

3 

41 


7 


7 


5 


70 


6 


-2 


4 

31 

= 7 

0 

+ 4 

1 

+ 9 

3 

122 


7 


7 


5 


13. 



3 

5' 


"*l' 


T 

1 

1 

, X = 

*2 

. b = 

-1 

5 

4 


*3 


0 


0 

-3 

l" 


■*l' 

1 

0 

-8 

, X = 

x 2 

5 

9 

- 1 

x 3 

3 

- 1 

7 


*4 


1 

3 

0 

2 


16. 


-3 -15 -11 


(a) 

21 

- 15 

44 _ 


"4 - 

7-19 

-43 

(b) 

2 

2 

18 

17 


0 

5 

25 

35 


2 

3 

23 

24 


(c) 


3 3 

-1 4 

1 5 

4 -4 

0 14 


is a 2 x 3 matrix and 5^ is a 2 x 2 matrix. does not exist, 

(a) 


- 1 

23 

- 10 

37 

- 13 

8 

29 

23 

41 


an 0 0 0 0 0 

0 a.22 0 0 0 0 

0 0 (333 o 0 0 

0 0 0 3 44 0 0 

0 0 0 0 0 
0 0 0 0 0 a 66 


21. 

(a) 




















































(b) 


(c) 


(d) 


an 

■312 

a 13 

0 

®22 

<323 

0 

0 

13 33 

0 

0 

0 

0 

0 

0 

0 

0 

0 

an 

0 

0 

a 2\ 

fl 22 

0 

a 3 1 

fl 32 

a 33 

a 4 1 

a 42 

fl 4 3 

an 

a 52 

a 53 

&6\ 

a 62 

«63 

an 

fl 12 

0 

a 2\ 

a 22 

fl 23 

0 

a 32 

£333 

0 

0 

£343 

0 

0 

0 

0 

0 

0 


a 14 

a 15 

a \6 

3 24 

&25 

&26 

3 34 

<23 5 

&36 

£344 

£345 

«46 

0 

£3 55 

«56 

0 

0 

a 66 

0 

0 

0 

0 

0 

0 

0 

0 

0 

£344 

0 

0 

£3 54 

£3 55 

0 

3 64 

^ 65 

a 66 

0 

0 

0 

0 

0 

0 

£3 34 

0 

0 

£344 

£345 

0 

£354 

£3 55 

a 56 

0 

^ 65 

a 66 


27 . 110 
One; namely, A = 1 — 1 0 

0 0 0 


30 . 


(a) 


Yes; for example. 


0 1 
0 0 


^ Yes; for example, 


1 0 
0 0 


True 

(a) 


^ False; for example, A = 


True 

(c) 


True 

(d) 


Exercise Set 1.4 (page 48) 
















1 _ 

2 - r 

. B~ l = 

1 

_i 

, c~ l = 

~ 1 _ 2 

2 

. D~ l = 

1- 

O 


_ -5 3_ 


i 

i— | '-n 

1 

_i 


1 3_ 


_° i 


7. 



5 

1 

13 

13 

3 

2 

13 

13 



1 

3 

7 



1 1 
5 5 



9 

1 

13 

13 

2 

6 

13 

13 


9. 


(a) 


p(A) = 


1 

1 


( 5) *00 = 


20 7 
14 6 


(c) 


p(A) = 


39 13 
26 13 


11 . 


cos# — sin# 
sin # cos # 


13. 




1 

ail 

0 


0 

1 

a 22 


0 

0 


0 0 


1 


18. C= -A~ X BA~ { 































19 


(a) 


0 


0 


1 

2 

1 

2 


1 

2 


0 


0 


1 



1 

2 

1 

2 


(b) 


1-10 
0 1 0 

0 0 1 

0 0 0 


0 

0 

-1 
1 


20 . 


(a) One example is 


(b) One example is 


1 

2 

3 

0 

1 

1 


2 3 
1 4 
4 5 

- 1 
0 
1 


- 1 
- 1 
0 


22. Yes 



1 

2 

1 

2 

1 

2 


33. 


i 1 0 0 

Oil 0 
0 Oil 


If A is invertible, then is invertible, 
(a) 


True 

(b) 


Exercise Set 1.5 (page 57) 


1. (a), (c), (d), (f) 














3 . 



0 

0 

1 


(a) 

0 

1 

0 



1 

0 

0 



'0 

0 

f 


(b) 

0 

1 

0 



1 

0 

0 




1 

0 

O' 

(c) 


0 

1 

0 



2 

0 

1 


1 

0 

O' 


(d) 

0 

1 

0 



2 

0 

1 




7 


4' 

(a) 


2 

— 

1_ 



5 


2 

(b) 


39 


13 



4 


1 



39 


13 


Not invertible 
(c) 


8 . 



1 3 

r 

(a) 

0 1-1 


-2 2 

0 

(b) 

1 

-3^2 

26 


4^2 

26 

J1 

26 


0 

0 

(c) 

1 

1 

0 

1 


3 

3 


O 

1 


0 

0 - 


Not invertible 
(d) 


4 3 1 1 

5 5 5 5 

f 0 -1 0 

1 0 0 0 

4 2 _1 _1 

5 5 5 5 


(e) 

























10. 


( a ) E x = 


(b) 


"l o" 

II 

C-l 

o 

1_ 

_5 1_ 

0 | 



2 


a~ 1 = e 2 e x 


(c) 


A = Br l B? 1 


11 . 


(a) 


(b) 


(c) 


1 -4 7 

4 5-3 

2-1 0 


2 

4 

3 
1 

10 

4 
1 


■ 1 

5 

3 

-4 

9 

5 

-4 


0 

- 1 
7 

— 6 
-3 
7 


14. 


'0 

1 

o' 

1 

0 

O' 

'1 

0 

o' 

'1 

3 

3 

8' 

1 

0 

0 

0 

1 

0 

0 

1 

0 

0 

1 

7 

8 

0 

0 

1 

-2 

0 

1 

0 

1 

1 

0 

0 

0 

0 


Add -1 times the first row to the second row. 

(b) 

Add -1 times the first row to the third row. 
Add -1 times the second row to the first row. 
Add the second row to the third row. 


24. In general, no. Try b=\,a = c = d = 0- 


Exercise Set 1.6 (page 66) 

1- XI = 3, x 2 = - 1 

4- *1 = 1, *2= — IT *3 = 16 


6. w = — 6. x = 1, y = 10> z = —7 




















9 


(a) Xl ~ 3* ’ * 2 ~ 3’*2- V 


(b) T1 - “ 3^2- 3> * 3 - 3° 

x 1 = 3> X'i = 0 ? X2 = — 4 

(c) 


11 . _ 22 _ 1 

(a) 17’* 2- 17 


21 


11 


(b) 


* 1 = W’* 2= 17 


13. 


(a) 7:1 “ 15’ * 2 “ 15 


34 


28 


(b) 


* 1 = T5’* 2 = 'l5 


19 


13 


(c) 


* 1 = TT’* 2= 15 


(d) 


*1 = -5^2 = j 


*1 = — 12 — X 2 — —5 — £’X2 — t 
(a) 


*1 = 7 — 3u xi = 3 — t, x^ = t 
(b) 


19. bi = £3 + &4’ b'i = 2i 3 4- 


21. 

11 

12 

-3 

27 

26 

X = 

-6 

-8 

1 

- 13 

- 17 


- 15 

-21 

9 

-38 

-35 


Only the trivial solution ^ = x 2 = X 2 = x^ = ^i\ invertible 

(a) 

Infinitely many solutions; not invertible 

(b) 




28. 


/ — A is invertible. 


(a) 


(b) 


K= (/-J) -1 !) 


30. Yes, for nonsquare matrices 

Exercise Set 1.7 (page 73) 


(a) 

(b) 

(c) 


2 ° 
° “I 


Not invertible 


-1 0 0 
0 1 0 

0 0 3 


3. 

'l 0" 
0 4 


'l 0" 

- 

(a) A 2 = 

. ^ 2 = 

0 j- 

, ,4“* = 




4 

_ 


(b) 


(a) 

7. a = 2, b = - 1 


1 


0 


0 1 / ( — 2) J 



7 0 

0 





4 

0 ? 


'4 

0 

0 ' 


a 2 = 

0 

, A~ 2 = 

0 

9 

0 

, A~ k = 


1 


0 

0 

16 



O 

O 





_ 


16 





2 k 0 


0 

0 


0 

0 


0 4' 


10. 


(a) 


(b) 


1 
0 
0 

0 

0 


0 - 1 
0 

1 

2 

0 






















11. 


16. 


(a) 


"ail 

312 

313 " 

"3 

0 

0" 

321 

322 

323 

0 

5 

0 

331 

332 

a 33 

0 

0 

7 


No 


(b) 


Yes 


(b) 


17. Yes 


19. 


l 

o 

o 


I 

o 

o 


o 

o 
_1 


0 4 0 


0 4 0 


0-10 


0 0 4 


0 0-1 


0 0 4 



I 

o 

o 

1 

1_ 


I 

1 

o 

o 
_1 


1 

o 

o 


0-10 

? 

0 4 0 


0-1 0 

? 

0 0 4 


0 0-1 


0 0-1 



- 1 
0 
0 


-10 0 
0 4 0 
0 0 4 

0 0 
-1 0 
0 - 1 


Yes 

(a) 


(b) 


No (unless « = 1) 


Yes 


(c) 


(d) 


No (unless n = \) 


7 i 1 

(a) *l = 4’*2=l'*3=-2 


(b) 


*1 = — 8. X2 = — 4 ? *3 = 3 


25. 


A = 


1 10 
0 -2 


§(!+*) 


Supplementary Exercises (page 76) 
























X 1 = I* + Jy , / = = y* + -|y 


3 One possible answer is 


— 2x2— x 3~ * 4=0 
*1 4 5^2 4 2^4 = 0 


5. x = 4> y = 2, z = 3 


a ^ 0> b ^ 2 
(a) 


a * 0. b = 2 

(b) 


a — 0, b — 2 

(c) 


a — 0> b ^ 2 

(d) 



0 2 

1 1 


11 . 


(a) 

(b) 

(c) 


X = 

X = 

x = 


-1 3 

6 0 

1 -2 
3 1 

113 

37 

_ 20 

37 



160 

37 
46 
' 37 


mpn multiplications and m p (« — 1) additions 


15. a = l,b= -2,c = 3 


16. a—\, b — —A,c— —5 


16 . ^_ 2 B- — 

5 5’ 













29 


if a ^ c 


(b) 


a n 

0 

0 ' 

1 

0 

b n 

0 

, where d = ( 

d 

0 

c n 

1 


a n — c H 




H 1 if a — c 


Exercise Set 2.1 (page 94) 


M ii = 29, M i 2 = 21i M\ 3 = 27, M.2\ = — ll’Af 22 = 13’^23 = — 5i Af 3 i = — 19, ^32 = — 19, 
< a ) M 33 = 19 


(b) 


C 11 = 29, C 12 = - 21, C 13 = 27, C 2 i = 11, C 22 = 13, C 23 = 5, C 31 = - 19, C 32 = 19, C 33 = 19 


3. 152 


4. 


(a) adj(j4) = 


(b) 


A-' = 


29 11 -19 

-21 13 19 

27 5 19 

29 11 


152 152 

21 13 


152 152 

27 5 


152 152 


19 

152 

19 

152 

19 

152 


6. -66 


8 . k 2 -Sk 2 - 10t f 95 


11 . 


A~' = 


3 -5 -5 
-3 4 5 

2 -2 -3 


13. 


A~ L = 


1 I 1 

0 1 f 
0 0 1 


15. 


A~' = 


-4 3 
2 -1 
-7 0 
6 0 


0 

0 

- 1 


- 1 
0 
8 

1 -7 
























16. x i = 1> X2 = 2 



61 z _46 
55’ 11 


2j Cramer's rule does not apply. 


cos# — sin# 0 
A * = sin# cos 0 0 

0 0 1 


24. x — y — 0, z — 2, w — 0 
31. det(j4) = 10 x ( - 108) = - 1080 

34. One 


Exercise Set 2.2 (page 101) 


-30 

(a) 


-2 

(b) 


0 

(c) 


0 

(d) 


4. 30 

6 . -17 

8 . 39 


11 . -2 





12 . 


-6 


(a) 


72 

(b) 

-6 

(c) 


18 

(d) 


det(J) = - 1 
(a) 


det(j4) = 1 

(b) 


18 ‘ * = 0 ,- 1 , ^ 


Exercise Set 2.3 (page 109) 


det(2J) = — 40 = 2 2 det(j4) 

(a) 

det( - 2A) = - 448 = ( - 2 ) 3 det(^) 

(b) 


Invertible 

(a) 


Not invertible 

(b) 


Not invertible 
(c) 


Not invertible 

(d) 


6 If x = 0, the first and third rows are proportional. 

If x = 2, the first and second rows are proportional. 

t _l±j£L 
2 


12. 


(a) 



k= -1 


(b) 


14. 


(a) 

(b) 

(c) 


> 

1 

1 

["O 

_ 1 

■*r 


'O' 

. - 2 A “ 1 . 

x 2 


_0_ 

A — 2 -3' 

'*f 


'o' 

r-"i 

1 

< 

^r 

1 

x 2 


_0_ 

A — 3 -1 ' 

'*l' 


'o' 

5 A+ 3 

x 2 


0 


15. 


A — 2A — 3 = 0 


A = - 1, A=3 


u. 



" 


V 

iii. 

i 


t 


A — 5A — 6 = 0 


i. 


A = — 1, A= 6 


u. 


in. 


r _ 


3 , 

-i 


^rt 


4 

i 





t 


A^-4 = 0 


i. 


A = - 2, A = 2 


u. 


in. 


i ~ 


r 



— t 

5 




t 

t 




20. No 


AB is singular. 


Flase 


(a) 


22 . 
































True 


(b) 


Flase 

(c) 


True 

(d) 


True 

(a) 


True 

(b) 


Flase 

(c) 


True 

(d) 


Exercise Set 2.4 (page 117) 


5 

(a) 


9 

(b) 


6 

(c) 


10 

(d) 


0 

(e) 


2 

(f) 


3, 22 


5. 52 


7. a 2 — 5a + 21 


9 -65 


-123 


A = 1, A= -3 
(a) 


\= — 2> A = 3> A = 4 

(b) 


16. 275 

= -120 

(a) 

= -120 

(b) 

18. 3±j33 


22. Equals 0 if n > 1 


Supplementary Exercises (page 118) 


i- *'=§*+^,y= 




4. 2 


5. c 2 + a 2 -& 2 a 2 -|-& 2 -c 2 

cos =—s;—’ cos 1 =—si— 


12. det(5) = ( - !)”(”—O' 2 det(j4) 


The 7th and jth columns will be interchanged, 
(a) 


(b) 


The 7th column will be divided by c. 





_ c times the yth column will be added to the ith column. 


(c) 


A 3 -I- ( - a\\ - Ct 22 - fl33)A 2 

+ 0 3 ll fl 22 ^~ a U a 33 f fl22 i3 33 — fl 12 fl 21 — fl 13 l3 31 — fl 23 fl 32)^ 

+ ( fl ll fl 23 fl 32 +■ fl 12 fl 21 fl 33 + fl 13 fl 22 fl 31 “ fl ll fl 22 fl 33 — fl 12 fl 23 fl 31 “ fl 13 fl 21 fl 32) 


18. 

~ — 2^~ 


~5f 


' It 

(a) \ = — 5, A = 2, A = 4; 

£ 

9 

£ 

9 

m 


t 


£ 


t 


(b) 

A = l; 



Exercise Set 3.1 (page 130) 


(a) ^2= (-1,-1) 


(b) p l p 2 = C-7. -2) 

(c) ^2 = (2,D 


( d ) ^2 =(«,*) 


(e) ^ = (-5,12, -6) 

(f) ^2 = ( 1 , - 1 , - 2 ) 


(g) P 1^2= (-<3, -<=) 


(h) = («, b, C) 


P{— 1,2, — 4) is one possible answer. 

(a) 

P(1 — 2 — 6) is one possible answer. 

(b) 


5 . 


















6 . 


(-2, 1, -4) 


(a) 

(-10, 6, 4) 

(b) 

(-7, 1, 10) 

(c) 

(80, -20, -80) 

(d) 

(132, -24, -72) 

(e) 

(-77, 8, 94) 

(f) 


ci = 2> C2 = — 1 ’ C2 = 2 


ci=c 2 = c 3 = 0 


*' = 5,/ = 3 
(a) 


(b) 


x — — b y = 3 


(4,1 ] 

r=Ll-4' |» + V = 

T- 1 

1 


I 2 2 ) 

’ l 2 2 ’ 

2 

2 ’ y 2 ’ 2 j 


Exercise Set 3.2 (page 134) 

1. 5 

(a) 



5 

(c) 


(d) 


2^3 









(e) 




(f) 


6 


i/S3 

(a) 


JTf + 1 / 2 ? 

(b) 


(c) 


4/l7 



(e) 


(^i" Jr' /k) 


(f) 


1 


(b) 


3 A 
5’ 5 


(c) 


2 

7’ 


1 i 
7' 7 


A sphere of radius 1 centered at (xo, yo, zq) 


Ct=C = 0 

(a) 


(b) 


At least one of a or c is not zero, that is, a 1 + c l > 0 


17. 


(a) 


The distance from x to the origin is less than 1. 


\\x -*oll > 1 
(b) 


Exercise Set 3.3 (page 142) 






1 


-11 


(a) 

-24 

(b) 

0 

(c) 

0 

(d) 


Orthogonal 

(a) 

Obtuse 

(b) 

Acute 

(c) 

Obtuse 

(d) 


( 6 , 2 ) 


(a) 


(b) 1 

f 21 14 \ 

i 13’ 13 j 

(c) 1 

(— 1 -HI 

^ 13 ’ ’ 13 J 

(d) 1 

{13 12 32 

\B9' 89’ 89 j 

(3£ 2jfc) for any scalar /c 
(b) 

(c) 1 


11. 

cos 9\ = 



cos 6*3 


13. ± (l / ^3, 1 / ^3, - 1 / /3) 




16. 


20 . 


21 . 


27. 


10 

(a) 3 

_ 6 

(b) 5 

-60 4 34^3 

(c) 33 

1 

(d) 2 


COS 


-1 




(b) 


cos 3 = 


- 7 cos 0 = 


c 

Ivl 


The vector u is dotted with a scalar. 

(a) 

A scalar is added to the vector w. 

(b) 

Scalars do not have norms. 

(c) 

The scalar k is dotted with a vector. 

(d) 


No; it merely says that u is orthogonal to v — w- 

30. r = (11 - r) 4 (v - r) —4 (w - r) —^ 
ll«ll 2 IMI 2 M 2 

31 Theorem of Pythagoras 

Exercise Set 3.4 (page 153) 

(32, -6, -4) 

(a) 


(-14, -20, -82) 


(b) 








(27, 40, -42) 


(c) 

(0, 176, -264) 

(d) 

(-44, 55, -22) 

(e) 

(-8, -3, -8) 

(f) 


J59 

(a) 


(b) 


^ToT 


(c) 


0 


For example, (1, 1, 1) x (2, - 3, 5) = (3, - 3, - 5) 


-3 

(a) 

3 

(b) 

3 

(c) 

-3 

(d) 

-3 

(e) 

0 

(f) 


No 

(a) 


Yes 

(b) 


No 


(c) 



i3. / 6 _ 4_] (__6 _ 3_ 4 } 

fii’feT’ /el j 


2 (v x u) 


i/26 

(a) 2 
J26 

(b) 3 


JU2 

(a) 

9 ?y40 o 19 / ' 

(b) 


m= ( 0 , 1 , 0 ) and n = ( 1 , 0 , 0 ) 
(a) 


(- 1 , 0 , 0 ) 

(b) 


(0 , 0 , - 1 ) 

(c) 


28. (-8, 0,-8) 


1 

(a) 3 

1 

(b) 2 


u - w 0 , v ■ w = 0 

(b) 


No, the equation is equivalent to u x (v — w) = 0 and hence to v — w = jfcu for some scalar k. 


38. The are collinear. 


Exercise Set 3.5 (page 162) 











-2(x 4 1)4 (y-3)-(z + 2) = 0 

(a) 


Or — 1) 4- 9(y — 1)4 8(z-4) = 0 

(b) 

2z = 0 

(c) 


x + 2y | 3z = 0 

(d) 


(0, 0, 5) is a point in the plane and n =(_ 3,7,2) is a normal vector so that 

( a ) _ 3(x — 0 ) -|- l(y — 0) + 2(z — 5) = 0 is a point-normal form; other points and normals yield other correct 
answers. 

(* - 0 ) 4 0 (y - 0 ) - 4(z - 0 ) = 0 is a possibility 

(b) 


Not parallel 
(a) 


Parrllel 

(b) 


Parallel 

(c) 


i = 34 2 t, y = -\-\-t,z = 2-\- 31 

(a) 


x = — 2 4 6 £, y — 3 — 6t, z — —3 — 2 1 

(b) 


x = 2, y — 2 -\- L z = 6 

(c) 


x = U y = — 2t,z=3t 

(d) 


x — -12-7 £,y= -41-23 t,z = t 
(a) 


( b) *=2 t ’y = 0’Z=‘ 


Parallel 

(a) 


13. 


Not parallel 


(b) 


17. 2x +- 3y - 5z h 36 = 0 


z — zq — 0 

(a) 


= 0 

(b) 


y -^0 = 0 

(c) 


21. 5x-2y+z-34 = 0 
23. y 4 2z — 9 = 0 


27. * + 5y + 3z- 13 = 0 


29. 4x + I3y-z- 17 = 0 


31. 3x — y— z — 2 = 0 


* = 11 + JZ_, y= _£L_X, _ 

(a) 23 23 y 23 23 z 


(b) I= “5^ = 0 ' z = ' 


5 

(a) 3 

1 

(b) fis 

4 

(c) 


■*^ 3 _ 1 = ^ 2 . 

(a) 2 * + 3 

x + 2 _ _ y-3 _ _ z± 3 

(b) 6 6 2 








44. 


x — 2y — 17 = 0 and x + Az — 27 = 0 is one possible answer. 


(a) 

x — 2v = 0 and — 7v -|- 2z = 0 is one possible answer. 

(b) 


9 k 35° 
(a) 


(b) 


47. They are identical. 


Exercise Set 4.1 (page 178) 


(-1,9,-11,1) 

(a) 

(22, 53, -19, 14) 

(b) 


(-13, 13, -36, -2) 
(c) 


(-90,-114, 60, -36) 

(d) 

(-9, -5, -5, -3) 

(e) 


(27, 29, -27, 9) 

(f) 


= 1-^2= 1' C3 = — 1, C4 = 1 


i/29 

(a) 


3 

(b) 

13 


(c) 



f_J_3_\ (__1_ 3 ) 

<a) [{w' /to' fio) 


Yes 

(a) 


No 

(b) 


Yes 

(c) 


No 

(d) 


No 

(e) 


Yes 

(f) 


* = -3 
(a) 


k= -2,k = -3 

(b) 


19. *i = T *2 = —1^3 = 2 


The component in the a direction is proj a ii 


23. The do not intersect. 


Euclidean measure of “box” in R n \ 


= 4L( _ 1, 2, 3)i the orthogonal component is -j-y (34, 11, 52, 


a\a 2 --£t n 


(a) 

(b) Length of diagonal: ^2 + ~2 ~ ~ ~2 


27). 







35. 


d (u, v) = ^2 


(a) 


True 

(a) 


True 

(b) 


False 

(c) 


True 

(d) 


True, unless u = 0 
(e) 


Exercise Set 4.2 (page 193) 


Linear; pp _» p 2 

(a) 

Nonlinear; p 2 _* p} 

(b) 


Linear; ^3_ + ^3 

(c) 

Nonlinear; p 4 _, p} 

(d) 


4 -1 
3 2 


- 1 
1 

- 1 


;7X-1,2,4) = (3, -2, -3) 


5. 


(a) 


0 1 

- 1 0 

1 3 

1 - 1 


7 

2 

- 1 

1 

0 

1 

1 

0 

1 

0 

0 

0 


(b) 












0 

0 

0 



0 

0 

0 



0 

0 

0 



0 

0 

0 



0 

0 

0 



0 

0 


0 

1 

1 

0 


0 

0 

0 

0 


1 

0 

0 

1 


0 

0 

1 

0 

— 

1 

0 


n- 1,4) = (5,4) 
(a) 


7(2,1, - 3) = (0, -2, 0) 
(b) 


(2, -5, -3) 

(a) 

(2, 5, 3) 

(b) 

(-2, -5,3) 

(c) 


i3. 

(a) | 


1 + 2^3 \ 

2 1 

(b) ' 

(0, 1, 2 / 2 ) 


( 

-1,-2, 2) 


(c) 



i5. 

(a) | 


- 1 + 2/3 \ 

2 ) 

(b) ' 

(-2/2, 1,0) 


(1, 2, 2) 


(c) 




17. 


(a) 


0 0 
1/2 -/3/2 












(b) 


(c) 


p p 

p p 

1 0 
0 - 1 


19. 


(a) 


^ 3/8 

1/3 

0 


pi 16 

3/16 

1/8 



'0 


0 

0 

(b) 

0 


-1 

0 


0 


0 

-1 



0 

1 

0 

(c) 


0 

0 

-1 



■ 1 

0 

0 


1 /16 
-pi 16 
p/s 


21 . 


Yes 


(a) 


No 


(b) 


24. 


■ 4(1 — cos 0 ) 4- cos 9 l(l-co S 0 )--j=sm<? 4(1 — cos 9 ) — —)= sin# 

3 3 p 3 p 

4(1 — cos 9 ) — — 7 = sin# 4(1-CO S 0) + COS0 4(^ — cos3)- 7 = sin.@ 

3 p 3 3 p 

4(1 — cos 9 ) — — 7 = sin# 4(^ — cos*9) — — 7 = sin.^ 4(1 — cos 0 ) + cos & 

3 p 3 p 3 


90° 

(c) 


Twice the orthogonal projection on the a- ax is 
(a) 


Twice the reflection about the x-axis 


(b) 


The x-coordinate is stretched by a factor of 2 and the y-coordinate is stretched by a factor of 3. 
(a) 


(b) 


Rotation through 30° 














Rotation through the angle 29 


3< Only if £ = 0- 


Exercise Set 4.3 (page 206) 


Not one-to-one 
(a) 


One-to-one 

(b) 

One-to-one 

(c) 


One-to-one 

(d) 


One-to-one 

(e) 


One-to-one 

(f) 


One-to-one 

(g) 


3. For example, the vector (1, 3) is not in the range. 


(a) 


One-to-one; 


; T 


12 1 1 
jwi--w 2 , jwi F 


Not one-to-one 


(b) 


(c) 


One-to-one; 


0 - 1 

1 0 


; T = (-W7, -wi) 


Not one-to-one 


(d) 


Reflection about the x-axis 

(a) 

Rotation through the angle — - / 4 

(b) 






9. 


(c) 

(d) 

(e) 

(a) 

(b) 

(c) 

(d) 


Contraction by a factor of -j 


Reflection about the yz-plane 


Dilation by a factor of 5 


Linear 


Nonlinear 


Linear 


Nonlinear 


12 . 


For a reflection about the y-axis, 7(*i) = 


and T(e 2 ) = 


. Thus, T = 


-1 0 

0 1 


(b) p or a reflection about the xz-plane, J’(ei) = 

'l' 

0 

. r(e 2 ) = 

o' 
- 1 

, and TO 3 ) = 

'O' 

0 

. Thus, T— 


0 


0 


1 



For an orthogonal projection on the x-axis, T\y i) = 
(d) p or an orthogonal projection on the yz-plane, T(y\) = 


and T(e 2 ) - 


. Thus, T — 


1 0 
0 0 




L 

J L 


'O' 


'o' 


'o' 

0 

. rc«2 ) = 

1 

, and = 

0 

0 


0 


1 


. Thus, 


T = 


0 0 0 
0 1 0 
0 0 1 


^ For a rotation through a positive angle 0, T(e i) 

j,_ cos# — sin# 
sin# cos# 

W For a dilation by a factor k > 1, T(e i) = 


cos # 
sin# 


and 7 ’(o 2 ) 


- sin# 
cos 9 


. Thus, 


k 


'O' 


'O' 


O 

O 

0 

0 

> TM = 

k 

0 

, and T(e 3 ) = 

0 

£ 

. Thus, T = 

- 1 

0 

0 

0 0 

_1 


(a) T (*l) = 


and 7(e 2 ) = 


. Thus, T = 


-1 0 

0 0 


0 0 

-1 0 

0 1 


13. 


















































23. 


(b) n*i) = 

( c ) T (*0 = 


and T(e 2 ) = 
and 7(e 2 ) = 


1 .Thus, T 
0_ 

. Thus, T = 


0 1 

-1 0 


0 0 
3 0 


Linear transformation from p} _> p}\ one-to-one 


(a) 



Linear transformation from p? 
(b) 

| 

(a) 1 

;h) 


(b) | 

<iE\ 

.4-4 


(c) 

1- 

*A. 

KJ~i 

-t* 1 

1 

19. 

■ol 



(a) A = 1; 


(b) A = 1; 


A= - 1 


A = 0 


A = 2; all vectors in p} are eigenvectors 


(c) 

(d) A=1; 



cos 29 

sin 29 


(a) 

sin 29 — 

cos 29 


(b) 

f l + 5|/3 

f3- 

5 \ 

l 2 ^ 

2 



The range of Tis a proper subset of P n . 
(a) 


(b) 


T must map infinitely many vectors to 0. 


Exercise Set 4.4 (page 217) 

































x 2 + 2 x- 1 — 2(3* 2 + 2)= — 5x 2 4 2x — 5 
(a) 1 > 


4. 


(a) 


Yes ;A = 


1 0 0 
0 1 0 
0 0 1 
0 0 0 


L\ P\ -* P\ wh ere L maps ax 4 b to (a \-b)x + a-b 

(a) 


3e f 4- 3s f = 6cosh(i) 

(a) 

Yes 

(b) 


12. y — 2x 2 


(a) 

y = 2x 3 -2x-h2 

(a) 

18. No, because of the arbitrary constant of integration 

(a) 

No (except for p n ) 

(b) 

Each Lj(x) is a polynomial of degree at most n and hence so is the sum -\ -1- _y H Z(x); also, 

p(x ;•) = 0 + 0 H-h 0 4 yj ■ Lj(xj) 4 0 H-f 0 + 0 =y v showing that this function is an interpolant of 

degree at most n. 

It is / H+lC = y where c is the vector of c ; - values andy is the vector of y-values. 

(b) 


Exercise Set 5.1 (page 226) 


Not a vector space. Axiom 8 fails. 





3. Not a vector space. Axioms 9 and 10 fail. 

5. The set is a vector space under the given operations. 
7. The set is a vector space under the given operations. 
Not a vector space. Axioms 1, 4, 5, and 6 fail. 

The set is a vector space under the given operations. 
13. The set is a vector space under the given operations. 
25 No. A vector space must have a zero element. 

26. No. Axioms 1, 4, and 6 will fail. 

29. Axiom 7 

1 . 

Axiom 4 

2 . 

Axiom 5 

3. 

Follows from statement 2 

4. 

Axiom 3 

5. 

Axiom 5 

6 . 

Axiom 4 

7. 

No; 0 \ = 0 1 + O 2 = 02 

Exercise Set 5.2 (page 238) 


1. (a), (c) 


3. (a), (b), (d) 


5. (a), (b), (d) 


Line; x = -^t,y = -h, z = t 
(a) ^ ^ 


Line; x — 2t, y — t, z — 0 

(b) 


Origin 

(c) 


Origin 

(d) 


Line; x — —3t,y— —2 t,z = t 
(e) 


Plane; x — 3y + z = 0 

(f) 


— 9 — lx — 15x 2 = — 2p i 4- p2 — 2p3 

(a) 

6+1 1a: -I- 6 x ^ — 4pi — 5p2 4-P3 

(b) 


o = Op 1 + Op 2 -I- Op 3 

(c) 

7 + St: + 9x 2 = Opi — 2p2 + 3p3 

(d) 


The vectors span, 
(a) 


The vectors do not span, 

(b) 


The vectors do not span, 
(c) 


The vectors span, 

(d) 


12. (a), (c), (e) 


y = z 


They span a line if they are collinear and not both 0. They span a plane if they are not collinear. 

(a) 

If u = a\ and v = £ u for some real numbers a, b 

(b) 

We must have = 0 since a subspace must contain x = 0 and then lj — 40 = ()• 

(c) 


26 

1 0 


0 1 


0 0 

For example, 

0 0 


0 0 

9 

1 0 ’ 


The set of matrices having one entry equal to 1 and all other entries equal to 0 

(b) 


Exercise Set 5.3 (page 248) 


U 2 is a scalar multiple of up 
(a) 


The vectors are linearly dependent by Theorem 5.3.3. 

(b) 

is a scalar multiple of p 1 . 

(c) 


B is a scalar multiple of A. 

(d) 


3 None 


They do not lie in a plane, 
(a) 


They do lie in a plane, 

(b) 


7. 2 3 7,3 7,2 

(b) v ‘ = T 2 ~ 7' 3 ' V2 = 2 vi + 2 V3 ’ ' 3 = “ 3 V1 + T 2 


a=-I,a=i 









If and only if the vector is not zero 


They are linearly independent since vi, v 2 , and V 3 do not lie in the same plane when they are placed with their 

(a) initial points at the origin. 

The are not linearly independent since vi, v 2 , and V 3 lie in the same plane when they are placed with their 

(b) initial points at the origin. 


20 . (a), (d), (e), (f) 


False 

(a) 


False 

(b) 


True 

(c) 


False 

(d) 


Yes 

(a) 


Exercise Set 5.4 (page 263) 


A basis for p} has two linearly independent vectors. 

(a) 

A basis for p 3 has three linearly independent vectors. 

(b) 


A basis for p 1 has three linearly independent vectors, 
(c) 


A basis for M 22 h as four linearly independent vectors, 

(d) 


3. (a), (b) 


(w)s=(3, -7) 

(a) 


(b) •") s (28' 14) 

<0 Mr {“■^) 


<?)*=(. 3 , - 2 , 1 ) 

(a) 


(?) s =(-2, 0 , 1 ) 
(b) 


11 . (^=(-1,1, -1,3) 


Basis: ^ 1, 0 j, (0, -1, 0, 1); dimension = 2 


Basis: (3, 1, 0), (-1, 0, 1); dimension = 2 


3-dimensional 

(a) 


2-dimensional 

(b) 


1-dimensional 

(c) 


20 . 3-dimensional 


, x {vi,V2, ei) or { V 1 ,V2, e 2 ) 

(a) 


„ x {v 1 , v 2 , <?l) or {vi, v 2 , <? 2 ) or {vi, v 2 , * 3 ) 
(b) 


27 . 

(a) 

One possible answer is < 

f — 1 4 - x — 3 + 3* 

(b) 

One possible answer is < 

f 1 f x, x 2 , — 2 4- 2x 2 j 

(c) 

One possible answer is < 

{l+x-3* 2 }. 


6* 2 , 9 



29. 


( 2 , 0 ) 

(a) 

(b > fr'/f) 

( 0 , 1 ) 

(c) 

< d > 


31. 


Yes; for example. 


1 

o 
_1 


I 

o 

_1 

1 

-H 

O 

_1 


1 

o 

-H 


n 

(a) 


n{n + 1) / 2 

(b) 


n{n h 1)/2 

(c) 


The dimension is « _ 1. 
(a) 


(b) 


(1, 0, 0, , 0, -1), (0, 1, 0, ... , 0, -1), (0, 0, 1, ... , 0, -1), ... , (0, 0, 0, ... , 1, -1) is a basis of size n . 


Exercise Set 5.5 (page 276) 


1 . 

2 


-1 


0 


1 

i-l = (2, - 1, 0, 1), i’2 = (3, 5, 7, - 1), 1-3 = (1, 4, 2, 7); ci = 

3 

» f 2 = 

5 

» '-3 = 

7 

, e 4 = 

- 1 


1 


4 


2 


7 



(b) 


b is not in the column space of A. 


1 


- 1 


1 


5 

9 

-3 

3 

+ 

1 

= 

1 

1 


1 


1 


- 1 


(c) 


























(e) 



2 


1 


- 1 


1 

(d) 

0 

= 

1 

+ (*-!) 

1 

+ ^ 

- 1 


0 


- 1 


- 1 


1 


= -26 


1 


2 


0 


1 

0 


1 


2 


1 

+ 13 

-7 

1 

+ 4 

1 

2 


3 

0 


1 


2 


2 



"f 


'3' 


'3' 

(a) 

0 

+ t 

1 


1 


(b) 

1 

1 

O l\J 
_1 

+ t 

' - r 
-1 


' - r 
-1 


0 


i 


i 


-1 


2 


- 1 


-2 


2 


- 1 


-2 

0 


1 


0 


0 


1 


0 


0 

0 

+ r 

0 

+ s 

l 

+ t 

0 

\r 

0 

4- s 

1 

+ t 

0 

0 


0 


0 


1 


0 


0 


1 


6 


1 


1 


1 


1 

5 


5 


5 


5 


5 

7 


4 


3 


4 


3 

5 

+ s 

5 

1 t 

5 

;s 

5 

+ t 

5 

0 


1 


0 


1 


0 

0 


0 


1 


0 


1 


7 . 


(a) 


ri = [1 


(b) 

ri = [1 





T 


' 2 ' 

0 2 ] * r 2 = 

[0 

0 l],c = 

0 

> = 

1 




0 


0 






T 

— 3 0 0] 

’ 1’2 

= [010 

0] 

’ Cl = 

0 

0 


0 


-3 

1 

0 

0 


(c) 


i'i = [l 2 4 5].i’2=[0 1 


-3 0],r 3 =[0 0 1 


— 3]> i' 4 = [ 0 0 0 1 ], ci 


a' 


5" 

-3 


0 

1 

, c 4 = 

-3 

0 


1 

0 


0 


(d) 

i'l = [ 1 2 -1 5].r 2 =[0 1 4 3 ] ’ r 3 = [ 0 0 1 — 7 ] > r 4 = [ 0 0 0 l].ci 


V 


" 2 ' 

0 


1 

0 

’ <7 = 

0 

0 


0 

1 

o 

_1 


1 

0 

1_ 


T 


' 2 ' 

0 


1 

0 

’ = 

0 

0 


0 



















































































' - l" 


5' 


4 


3 

03 = 

1 

, c 4 = 

-7 


0 


1 


9 . 


(a) 


(b) 


1 

5 

7 

2 

4 

0 



(c) 


(d) 



4 

1 

3 

4 

-2 

0 

3 


(e) 


-3 

3 

— 3 
-6 
9 


2 

6 

-2 

0 

2 


(a) (1, 1, -4, -3), (0, 1, -5, -2), (0, 0, 1, - IJ 

(b) (1,-1, 2,0), (0,1, 0,0), (0,0, 1. 

(1,1,0, 0), (0,1, 1,1), (0, 0,1,1), (0, 0, 0, 

(C) 


0 0 0 
(b) 0 10 
0 0 1 


17 . 


3 a 
3 b 


— 5a 
-5b 


for all real numbers a, b not both 0 . 


Exercise Set 5.6 (page 288) 


Rank (21) = rank ^4 ^ J = 2 






























3 . 


2;1 


(a) 

l; 2 

(b) 

2 ; 2 

(c) 

2; 3 

(d) 

3: 2 

(e) 


Rank = 4> nullity = 0 

(a) 

Rank = 3, nullity = 2 

(b) 

Rank = 3, nullity = 0 

(c) 


Yes, 0 

(a) 

No 

(b) 

Yes, 2 

(c) 

Yes, 7 

(d) 

No 

(e) 

Yes, 4 

(f) 

Yes, 0 

(g) 


9. b i = n &2 = £’ &3 = 4s — 3r> = 2r — s> b$ = 8s — Ir 


No 


Rank is 2 if r = 2 and s = 1; the rank is never 1. 


16 . 


(a) 


1 0 0 
0 1 0 
0 0 0 


A line through the origin 

(b) 


(c) 


A plane through the origin 


(d) 


The nullspace is a line through the origin and the row space is a plane through the origin. 


3 

(a) 


5 

(b) 

3 

(c) 


3 

(d) 


Supplementary Exercises (page 290) 


All of R 3 
(a) 


Plane: 2x — 3y -+ z = 0 

(b) 


Line: x — 2t, y — t, z — 0 
(c) 


The origin: (0, 0, 0) 

(d) 


< 4 , 1 , 1 )+ 6 ( 0 , - 1 , 2 ) 

(a) 


(a fc)(3, — 1, 2) +&(1, 4, 1) 

(b) 




a(2, 3, 0) +*(-1, 0,4) 4 c( 4, -1,1) 


(c) 


(a) v =(—l +'■)'•, f 


7 , No 


Rank = 2, nullity = 1 

(a) 


Rank = 3, nullity = 2 

(b) 


"Rank- = n 4- 1, nullity = « 
(C) 




2 

(a) 


1 

(b) 

2 

(c) 


3 

(d) 


Exercise Set 6.1 (page 304) 


2 

(a) 


11 

(b) 


-13 

(c) 


-8 

(d) 


I rv' 3 ; r arbitrary 


1 . 


0 


(e) 


3 

(a) 


56 

(b) 


29 

(b) 


7 . 


(a) 

(b) 


0 

0 

2 0 ' 
0 


No. Axiom 4 fails. 

(a) 

No. Axioms 2 and 3 fail. 

(b) 

Yes 

(c) 

No. Axiom 4 fails. 

(d) 


3t[2 

(a) 



(c) 


3/13 


Jl4 

(a) 

0 

(b) 






15. 


(a) 

(b) 


/l05 

^47 



2 

(b) 3 


fe 


19. , . 1 , 

{«. v } = 3«ivi 4- u 2 V2 


No for Py since p = x pr — ^ j(Y — 1) satisfies (p, p J 


_28 
(a) 15 


0 

(b) 


34. fl = 1/25,6 = 1/16 


Exercise Set 6.2 (page 315) 


Yes 

(a) 


No 

(b) 


Yes 

(c) 


No 

(d) 


No 

(e) 


Yes 


(f) 



5 . 


1 

(a) ^2 

3 

(b) y/73 
0 

(c) 

20 

(d) 9^10 
1 

(e) /2 
2 

(f) /55 


Orthogonal 

(a) 


Orthogonal 

(b) 


Orthogonal 

(c) 


Not orthogonal 

(d) 


=1 ^{-34,44, -6,11) 


x — Uy — —2 Uz= —31 
(a) 


2x - 5y 4- 4z = 0 

(b) 

x — z = 0 
(C) 


17. 

1 


0 


2 

(a) 

3 

? 

1 

? 

- 1 


1 


1 


1 


( 16 , 19 , 1 ) 
(a) 


18 . 









w CO. 1, Q). (i 0.1 


(-1,-1, 1,0), 0, l) 


(-1, -1, 1, 0, 0), (-2, -1, 0, 1, 0), (-1, -2, 0, 0, 1) 

(d) 


32. 


{«. v ) = 2 U1V1 + e ll2V2 


35. 


The line y — — x 


The xz-plane 


(b) 


The a- ax is 


(c) 


37. 


False 


(a) 


True 

(b) 


True 

(c) 


False 


(d) 


Exercise Set 6.3 (page 328) 

1 . (a), (b), (d) 

3. (b), (d) 

5. (a) 


7 . 






(C) 






■0 . 


^3’ ^3’ J\ ft 




7 ,1 

(a) “ 5 vi + 5 v 2 + 2 v 3 


37 9 

(b) “T V1 “ 5 V2 + 4v3 


3 1 5 

(c) - 7 V 1 - 7 V 2 -I 7 v 3 


11 . 


(a) 


(w)*=(-2/2,5/2) 


(b) 


(yr)s= (0. -2,1) 


13. 


(a) 


-f !)-(-*■ -f 


(b) 


|v|| = ^f\3, d (u, v) = 5^3, (^ v , v} — 13 


(b) ”=- 5 V1 “lo V2 h ° V3+ 2 V4 


<a) [kkh\\ kk°Hkk~k 

(b) (1. 0, 0), - 7=]. (»■ ) 


19. 


0 , 


J_2_^ f __2_1_^ 


21 . 


*'l = 




24 . 


(a) 


f 5 ~f 5 


f 5 J 5 









(b) 


(c) 


(e) 


(d) 


1 

& 

0 

1 

& 

1 

3 

_ 2 

3 

2 

3 

1 


1 

1 

1 

8 


1 


^2 3^2 

0 /3 


\f234 

11 

^234 

7 

\J234 

1 

i 
£ 
1 


1 

3 

^26 

3 


0 — 7 = -r= 


2 . 

£ 

J_ 

/6 


/2 |/2 

/ 2 ‘ 

0 /3 

1 

0 0 

4 


/ 6 _ 


J_ i/2 
/2 2/l9 

JL 

2/19 

3^2_ 

/l9 


1 

0 


3 

/l9 

'/2 

3 

f2 

/2 

3 

0 

\[\ 9 

3/2 

/l9 


|/l9" 

1 

0 

0 

1 

/l9 


/l9 


(f) 


Columns not linearly independent 


29. 


T ‘“ /2 >V2= /f ;[ ’ T3 -^'( 3 " 2_1 ) 


vi = 1, y 2 = ^3(2* - 1)’ v 3 = (^ 2 - ox + 1 j 

(l / \[ 5 , 1 / ^5 J, (2 / /30, - 3 / /30) 
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'21 

25' 

■*r 


' 20 ' 

25 

35_ 

_* 2 _ 


_20_ 


1. 


(a) 









































(b) 


15 

- 1 

5' 

■*r 


' - r 

- 1 

22 

30 

x 2 

= 

9 

5 

30 

45 

x 3 


13 


(a) 


1 . 


*1 = 5 ’ *2=2 


11 

2 

_ 9 
2 

-4 


(b) 


*1 


* 2 = - 


2 . 

3’ 


46 

21 

JL 

21 

11 

21 


(c) 


*1 — 12> X 2 — —3’X2 = 9i 


(d) 


x\ = 14» X 2 = 30, ^3 = 26; 


3 
3 
9 
0 

2 

6 

-2 

4 


(7, 2, 9, 5) 
(a) 


/_ 12 _ 4 12 16) 
(b) \ 5 5’ 5’ 5 ) 


7. 


(a) 


1 0 
0 0 


(b) 


0 0 
0 1 


vi = (2, -1,4) 
(a) 


4 

2 

8 

21 

21 

21 

2 

1 

4 

21 

21 

21 

8 

4 

16 

21 

21 

21 






















(C) 


21 Xo ~21 yo+ 21 Z ° 
2 1 4 

"IT * 0 4 IF 0 "IT * 0 

if* 0 - 2f ya 4 If* 0 


J491 
(d) 7— 


17. 


[P]=A T (AA T) j 


Since a T 0 = 0 

1 . 


Since j{ ^4 is invertible 

2 . 


Since the nullspace of A is nonzero if and only if the columns of A are dependent 

3. 
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(a) [ w ] s = 

(b) 

(e) [w] s = 


3 

-1 


28 


3 


14 


a 

h- 

a 


3. 


(a) (p)*=(4, -3,1), [v] s -- 


(b) 


(P)*=(0,2, -1), [p]^ = 


4 

-3 

1 

0 

2 

- 1 


w= (16, 10 , 12 ) 

(a) 


(b) 


<1 = 3 + 4x* 

















(C) 


5 = 


15 

6 


(a) 


(b) 


(c) 


13 
10 
_ 2 
5 

0 

-2 - 

[w] B - 


1 

2 

0 

_ 5 
2 

11 

2 


XL 

10 


» [w] b' = 


-4 

-7 


(a) 


(b) 


2 4 


-2 

5 

_ 7 
2 
23 
2 

6 


-3 

1 


11 . 


(b) 


(c) 


2 0 
1 3 


I 0 

1 1 
6 3 


(d) [h] B = 


2 

-5 


, [b] F ' = 
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(b) 


4 

5 


3 

5 


9 

12 

25 

25 

4 

3 

5 

5 

12 

16 

25 

25 


(a) 


1 0 
0 1 


3. 




























(b) 


(d) 


1 1 

y [2 /2 

1 1 

y [2 /2 

1 

fz 

1 2 


0 


1 

^3 


1 



(e) 


1 

2 

1 

2 

1 

2 

1 

2 


1 1 

2 2 

5 1 

6 6 

1 1 

6 6 

1 _ 5 

6 6 


1 

2 

1 

6 

5 

6 

1 

6 


(a) (-’ +3 ^ 3+ ^) 


<b> 


9. 





11 . 


12 . 




cos 0 0 

— sin# 

(a) 

,4 = 

0 

1 

0 



sin# 0 

cos 9 



'1 

0 

0 

(b) 

A = 

0 

cos 9 

sin 9 



0 

— sin# 

cos 9 


^2_ ^6_ _J2_ 

4 4 ~ 2 



^2 _^2 

4 4 2 














16. 


Rotation 


(a) 


Rotation followed by a reflection 


(b) 


20 . 


Rotation and reflection 


Rotation and dilation 

(b) 

Any rigid operator is angle preserving. Any dilation or contraction with £ * Q, 1 is angle preserving but not 

(c) rigid. 

22. a = 0, b = c = - { W3 or a = 0, b = - \j2T5, c - y[\i3 

Supplementary Exercises (page 356) 

(0, a, a, 0) with a * 0 




... , n 

rt 


0 approaches ^ 


12 . 


The diagonals of a parallelogram are perpendicular if and only if its sides have the same length. 


(b) 


Exercise Set 7.1 (page 367) 


l. 


A — 2A — 3 = 0 


(a) 







A 2 — 8A 4 16 = 0 

(b) 

A 2 - 12 = 0 

(c) 

A 2 + 3 = 0 

(d) 

A 2 = 0 

(e) 

A 2 — 2A 4 1 = 0 

(f) 


3. 


(a) Basis for eigenspace corresponding to A = 3- 

(b) Basis for eigenspace corresponding to A = 4- 


; basis for eigenspace corresponding to A = — 1: 


^ Basis for eigenspace corresponding to A = ^12: 

3 

{\2 

1 

There are no eigenspaces. 

(d) 

^ Basis for eigenspace corresponding to A = 0: 

^ Basis for eigenspace corresponding to A = 1: 


3 

y [\2 

1 


; basis for eigenspace corresponding to A = — y[\2 


"l" 


'O' 

_0_ 

? 

_1_ 

t 


'o' 

_0_ 

9 

_1_ 


1: basis 

'O' 

1 

; A = 2: basis 

1' 

2 

1 

1 

; A = 3- basis 

' - r 
1 


0 



1 


(b) 

I 

'._K j | 



1(15+5/2) 



1 

Ol — 

L-i-i 

1 

L-i-i 

^1 
_ 1 

A= 0: basis 

1 

3 

; A = 

^2: basis 

1(-1 + 2/2) 

; a= - 1/ 

2: basis 

i(-!-2/2) 


1 



1 



1 


- 

1' 






(c) 


6 







1 

6 

1 


A = — 8: basis 



































(d) 


A = 2- basis 


1 

3 

1 

3 

1 


(e) 


(f) 


A = 2- basis 


A = — 4: basis 


1 

3 

1 

3 

1 

-2 

8 

3 


; A = 3' basis 


(a) 

A = 4 

(b) 


10 . 


(a) 

(b) 

(c) 


A = - 1, A = 5 

A=3,A = 7, A= 1 
A= -y, A= 1, A = -l 


y — x and y = 2x 

(a) 

No lines 

(b) 


y = 0 

(c) 


-5 

(a) 

7 

(b) 


22. 

"1" 


" i" 

(a) 

> 

to 

II 

tojT- 

2 

Ai = i: 

0 

1 

1 




0 
















25. 


(b) 


Ai = — 2: 


(c) 


A! = 3 ; 


;a 2 = - 1 : 


"f 


" 1' 


2 

0 

;A2 = 4: 

1 

1 




0 


; A3 


; a 3 = 5 : 


A is 6 x 6 

(a) 


A is invertible, 

(b) 


A has three eigenspaces. 
(c) 
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1. A =0:1 or 2; A= 1:1; A = 2:l,2, or 3 


3 Not diagonalizable 


Not diagonalizable 


Not diagonalizable 


P = 


\ 0 
1 1 


■P~ l AP = 



11 . 

' -2 

0 

r 

i 

i— l 

ii 

'3 

0 

0" 

P = 

0 

1 

0 

0 

3 

0 


1 

0 

0 


0 

0 

2 


13. 

'1 

2 

r 

i 

1—^ 

k 

II 

'1 

0 

O' 

P = 

1 

3 

3 

0 

2 

0 


1 

3 

4 


0 

0 

3 


Not diagonalizable 


























17 . 


'l 

1 

0 

o' 


' -2 

0 

0 

o' 

0 

1 

1 

0 

1 

1 — L 
is 

II 

0 

-2 

0 

0 

0 

0 

1 

1 

0 

0 

3 

0 

0 

0 

0 

1 


0 

0 

0 

3 


19. 


- 1 10237 -2047 
0 1 0 
0 10245 -2048 


21 . 


A” = PD”P- 1 


1 1 1 
2 0-1 
1-1 1 


r o o 

0 3 H 0 
0 0 4” 


and \ 2 are as in Exercise 18 of Section 7.1. 


1 

6 

1 

2 

1 

3 


1 

3 

0 

1 

3 


1 

6 

1 

2 

1 

3 


One possibility is p = 


-h 
a — Ai 


-b 
a-X 2 


where Aj 


False 

(a) 


False 

(b) 


True 

(c) 


True 

(d) 


True 

(e) 


27. Eigenvalues X must satisfy — 1 < A < 1- 

(a) 

If A — PDP~^ with D diagonal, then lim A k — PD ! P 1 , where D' is obtained from D by setting all 

(b) * —» +o° 

diagonal entries that are not 1 to 0. 
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A 2 — 5A = 0; A= 0: one-dimensional; A = 5: one-dimensional 

(a) 

A 3 — 27A — 54 = 0; A = &’■ one-dimensional; A = — 3’- two-dimensional 

(b) 


1 . 
















'uoiH "^lh tnh 


\ 3 _ 3 \ 2 _ q; A = 3: one-dimensional; A = 0: two-dimensional 

(c) 

A 3 — 12A 2 -l- 36X - 32 = 0; A = 2: two-dimensional; A = 8: one-dimensional 

(d) 


A 4 — 8A 3 = 0; A = 0: three-dimensional; A = 8: one-dimensional 

(e) ' 

A 4 - 8A 3 -I- 22A 2 - 24A + 9 = 0; A = l: two-dimensional; A = 3: two-dimensional 

(f) 


3. 


P = 


£ 

P 



■P~ 1 AP = 

? 


3 0 
0 10 


5. 


P = 


4 

5 
0 
3 
5 



■P~ l AP = 

? 


25 

0 

0 



0 

0 


-50 




0 0 0 
0 3 0 
0 0 3 


9. 


P = 


i 1 

5 5 

1 i 

5 5 

0 0 


0 0 


0 0 



0 

3 
5 

4 

5 


■P~ X AP = 


- 25 0 0 0 

0 25 0 0 

0 0 -25 0 

0 0 0 25 


12 . 


(b) 



0 

1 

0 





















15 . 


Yes; take A — 


3 0 0 
0 3 4 
0 4 3 


Supplementary Exercises (page 384) 


The transformation rotates vectors through the angle 0; therefore, if 0 < 9 < x, then no nonzero vector is 
(b) transformed into a vector in the same or opposite direction. 


1 1 0 

(c) 0 2 1 

0 0 3 


9. 




15 30 
5 10 


75 150 
25 50 



375 750 
125 250 



1375 3750 
625 1250 


12 . 


(b) 


0 0 0 -1 
10 0 2 
0 10-1 
0 0 1-3 


13. The are all 0, 1, or -1. 


15. 


1 

1 

1 


0 

1 

2 

1 

2 


0 

1 

2 

1 

2 


Exercise Set 8.1 (page 398) 


3 Nonlinear 


Linear 


Linear 

(a) 


Nonlinear 

(b) 


















T{x\, xi) = y(3xi -X 2 , - i -4x2, 5*i -I- 10x2); ~ 3 ) — ~T~ ^F) 


15. T(x u x 2 , x 3 ) = ( - 41xi + 9x 2 4- 24x 3 , 14xi - 3x 2 - Sx 3 ); 7(7, 13, 7) = ( - 2, 3) 


Domain: R 2 ; codomain: R 2 ; (J 2 o T\) (x, y ) = (2x - 3y, 2x h 3y) 

(a) 

Domain: R 2 ; codomain: R 2 ; (T 2 o T{) (x, y) = (4x - 12 y, 3x - 9y) 

(b) 

Domain: R 2 ; codomain: R 2 ; (T 2 oT\)(x,y) = ( 2x 4- 3y, x — 2y) 

(c) 

Domain: R 2 ; codomain: R 2 ; (J 2 a T\)(x,y) = (0, 2x) 

(d) 


a \-d 

(a) 

(T 2 o T\) (j4 ) does not exist since T\ (ri) is not a 2 x 2 matrix. 

(b) 


(T 2 o T\) fag 4- a\x 4- a 2 x 2 ) = («0 +a\ + a 2 )x 4- (ct\ 4 2a 2 )x 1 4- a 2 x? 


(3T)(x\, x 2 ) = (6xi - 3x 2 , 3 x 2 4- 3xi) 

(b) 


No 

(b) 


x 2 4- 3x 

(a) 

sinx 

(b) 

e*-\ 

(c) 


Exercise Set 8.2 (page 405) 


1 . (a), (c) 


3 . (a), (b), (c) 


5. (b) 


(a) 


r 1 ) 


(1 -4 

(b) \ 2 - 


, l.oj 


No basis exists. 


(c) 


11 . 


(a) 


(b) 



'0" 


1 


0 


(c) 


Rank(T) — 1, nullity (T) — 2 


(d) 


Rank(j4) = l, nuUity(j4) = 2 


13. 


(a) 


" 

l" 


3 


- 1 

5 

2 


" - f 

r 

- 1 


1 

9 

0 


0 



0 

i ro 
_ 2 0 
7 ’ 0 
1 

14 

(b) _1 12 

0 
0 
1 

Rank(T) = 3» nuUity(T) = 2 

(c) 


(d) 


Rank(j4) = 3> nuUity(j4) = 2 


15. ker(T) = {0} ; R(T) = V 


















17. Nullity(7) = 0, rank(7) = 6 


21 . 


x = — t, y = =t,z = t, — oo < i < -1- do 


14* — 8y — 5z = 0 


(b) 


ker(-D) consists of all constant polynomials. 

ker(D o D) consists of all functions of the form ax A- b\ ker(D o D o D) consists of all functions of the form 


ax 2 -\- bx -\- c- 


30. 


Do Do Do D, where D is differentiation 


(a) 


Do Do ■■■ o D{n \-1 times) 


(b) 


Exercise Set 8.3 (page 413) 


ker(7) = {0} ; 7 is one-to-one. 


1 . 


(a) 



ker(7) = {0} ; 7 is one-to-one 

(c) 

ker(T) = {0} ; 7 is one-to-one 

(d) 

ker(T) = {*(1,1)} ; T is not one-to-one 

(e) 

ker (T) = {*(0, 1, - 1)}; 7 is not one-to-one 


T has no inverse. 


3. 


(b) 


*1 

x 2 

x 3 


(c) 


,-l 


*1 

x 2 

x 3 


1 1 3 

3 ^+g* 2 - 

i*H |*2+^*3 

3 5 1 

g*H s*2 + ? *3 

2*1 “ 2 * 2 + 2*3 

2 *‘+ 2*2 + I * 3 

2 *!+ 2*2 “ 2*3 


(d) T -l 



3x\ + 3x2— x 2 

x 2 

— 

— 2x\ — 2 x 2 + x 3 


*3 


— 4x\ — 5x2 + 2x2 


ker(T)= {«- 1, 1)} 

(a) 

T is not one-to-one since ker(T) * {0} • 

(b) 


T is one-to-one. 

(a) 

T is not one-to-one. 

(b) 

T is not one-to-one. 

(c) 

T is one-to-one. 

(d) 


a< 5 + 0 for i = 1 , 2, 3, ... , n 

(a) 

(b) T_1 C * :1 ' * 2 ’ * h) = {^ Xh ~k X2 ’ ~k x ”) 


(a) 7 , r 1 C^W) = j£ ^- ; 72 1 C/jW)= i pCi-l) ; C 7 ’ 2 ori) \p(x)) = ±p(x-l) 


( 1 ,- 1 ) 

(a) 

T~\2, 3) = 2 + x 

(d) 















17 . 


T is not one-to-one. 

(a) 


T is one-to-one. T 1 

a b 

c d 

= 

'a c " 

b d 


T is one-to-one. T _1 

a b 

c d 

= 

d 
— c 

-b 

a 


T is not one-to-one since, for example, / (x) = x 2 (x — l) 2 is in its kernel. 


Yes; it is one-to-one. 
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l. 


(a) 


0 0 0 
1 0 0 
0 1 0 
0 0 1 


1 -1 1 

(a) 0 1-2 

0 0 1 


5 . 


(a) 


0 0 


8 4 
3 3 


1 1 1 

(a) 024 
0 0 4 

3+ 10x+ 16x 2 

(b) 


9 . 


(a) IT(vi)] £ = 


> [T(y 2 )]b = 


(b) r(vi) = 



• T(J 2) = 


-2 

29 


























(C) 


7 


*1 

x 2 


(d) 


19 

7 

83 

7 


18 I 

*1 
x 2 

7 7 


7 7 

107 24 


^ [TCV1)]£ = 

"l‘ 


3' 


' - r 

2 

> [T(y2)] B = 

0 

• [^(V3)] 5 = 

5 


6 


-2 


4 


T(vi) = 16 4 51* + 19* 2 , T(v 2 ) = - 6 - 5* 4 5* 2 , T(v 3 ) = 7 4 40* 4 15* 2 
(b) 


(£) T[a 0 + ai x 4- * 2 ;r 2 ) = 239a ° ~ ^ + 289a 2 

20 Ian — 11 la i 4 247a 2 , 6 lan — 3 la i 4 107a 2 2 

4 8 * + 12 * 

T{ 1 4 * 2 ) = 22 4 56* ^ 14* 2 
(d) 1 ' 


13. 

'o 

o' 


1 

O 

O 

O 


'2 O' 

(a) r r-n „ , 

6 

o 

» \J2 ]b\B" — 

3 0 0 

> ITi \b",b = 

[T 2 O 7l ] = 

0 

0 

0 3 0 

0 0 

0 - 3 


1 

o 


0 0 3 




[T 2 o Ti] 5 ' b = 5 

(b) 


19 . 



'0 

0 


(a) 

0 

0 

— 


0 

1 



'0 

0 

O' 

(b) 

0 

1 

0 


0 

0 

2 


'2 

1 

O' 

(c) 

0 

2 

2 


0 

0 

2 

(d) 

14e 

2x 

— 8; 


2* 


20*V* since 


'2 1 O' 

4' 


14' 

0 2 2 

6 

= 

-8 

0 0 2 

- 10 


-20 


(a) 

B’,B ,U 

(b) 


21 






































22 . We can easily compute kernels, ranges, and compositions of linear transformations. 
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[T] b = 


1 -2 

0 - 1 


, [T] B ' = 


3 

56 

11 

11 

2 

3 

11 

11 


[T] b = 


1 

1 


13 

25 

1 

~f2 

1 

, [T] B ' = 

ll/2 

5 

ll/2 

9 

f 2 

f2_ 


ll/2 

ll/2 


5. 

1 

O 

O 


0 

0 

1_ 

[^] £ = 

0 1 0 

0 0 0 

, [T] b > = 

0 1 1 

0 0 0 


det(T) = 17 
(a) 


(b) 


det(7) = 0 


(c) 


det(T) = 1 


10 . 


(a) 


\T] b = 


11111 

0 2 4 6 8 

0 0 4 12 24 

0 0 0 8 32 

0 0 0 0 16 


, where B is the standard basis for p^: rank (T) = 5 and nullity (7) 


T is one-to-one. 


(b) 


< a > = 

(*>)«; = 


' - f 


V 


' - r 

1 

’ u 2 = 

0 

, U3 = 

-1 

0 


1 


1 

' - r 


V 


' - f 

1 

,U2 = 

0 

, U3 = 

-1 

0 


1 


1 


12 . 
































14 . 


«=)«; = 


"l" 


'O' 


' - f 

2 

’ u 2 = 

1 

, U 3 = 

0 

1 


0 


1 


(a) 


A = 1, A= -2, A= - 1 


^ Basis for eigenspace corresponding to A = l: 
basis for eigenspace corresponding to A = —2' 

basis for eigenspace corresponding to A = — 1: 


g — p~ 1 is similar to A. 

1. 

I = P~ l P 


The distributive law for matrices 


0 0 
0 1 


and 


2 3 
1 0 


-1 0 
1 0 

-2 1 ' 
1 0 


4 . 


5 . 


The determinant of a product is the product of the determinants. 
The commutative law for real multiplication 


det(f> _1 ) = 1 /det(P) 


The choice of an appropriate basis can yield a better understanding of the linear operator. 


Exercise Set 8.6 (page 445) 


2 , When A is noninvertible. 


No (not onto) 

(a) 


Yes 

(b) 


No (not one-to-one) 
(c) 
















No (not one-to-one) 


(d) 


11 . 


The matrix is 


0 

0 

0 

0 

0 

0 

0 

- 1 

0 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

-2 

0 

0 

0 

2 

0 


Supplementary Exercises (page 446) 


1. No. T’fxj -|- * 2 ) — 4(\i -1- X 2 ) 4 4- B) 4- (-4x2 4- B) — T(xi) 4- T(x 2 )’ an d if c * 1> then 

T(cx) = cj4x: I B * c(.4x -I- B) — cT{\)- 


T{* 3 ) and any two of T{* 2 )’ and T{* 4 ) form bases for the range; (- 1 , 1 , 0 , 1 ) is a basis for the kernel. 

(a) 

Rank = 3, nullity = 1 

(b) 


Rank (T) = 2 and nullity (T) = 2 
(a) 


T is not one-to-one. 

(b) 


Rank = 3» nullity = 1 


13 . 


10 0 0 
0 0 10 
0 10 0 
0 0 0 1 


Vi = 2ui 4- U2’ v 2 = — ill 4- u 2 4- U3, V3 = 3ui -I 4it2 I- 2«3 
(a) 


uj = — 2vi — 2\ r 2 4- V3’ 112 = 5vi 4- 4\'2 — 2\'3’ 113 = — 7vi — 5\'2 4- 3v3 

(b) 


17 . 


1-1 1 
0 1 0 
1 0-1 


[T] b = 








20 . 


(a) 


2 

6 

12 


- 3* 2 4- 3 

(d) 



2 The points are on the graph. 


24 . 


0 10 0 
0 0 10 
0 0 0 1 


0 

0 

0 


0 0 0 0 -1 

0 0 0 0 -0 


Exercise Set 9.1 (page 456) 


y\ =c\e 5x - 2 c 2S * 

(a) j, 

72 =c\e T C2 S 


— C23 2x T C2S 3x 
c\e x T 2c23 2x — C2&^ X 
2c23^ x — cys 3x 
e 2x — 2s ix 
e * = 2e 2x + 2e 3x 
— 2e 2x + 2s 3x 

7 . y = Cie 3 * f c 2 3~ 2x 


(a) 


(b) 


71 

72 

73 

71 

72 

73 


y = c\g x -T C2 .s 2x T C23 3x 


9 . 








Exercise Set 9.2 (page 466) 


l. 


(a) 


0 - 1 
-1 0 


(b) 


- 1 0 
0 - 1 


(c) 


1 0 
0 0 


(d) 


0 0 
0 1 


3 . 


(a) 


1 0 0 

0 1 0 

0 0-1 


(b) 


0 0 

-1 0 

0 1 


(c) 


-10 0 
0 1 0 

0 0 1 


5 . 


(a) 


-1 0 

0 0 

0 1 


(b) 


1 0 0 

0 0-1 

0 1 0 


(C) 


0 0 1 

0 1 0 

-10 0 


Rectangle with vertices at (0, 0), (-3, 0), (0, 1), (-3, 1) 


9 . 


(a) 


1 0 
4 1 


(b) 


1 -2 

0 1 


10 . 


(a) 


(b) 


1 0 

i. 

6 O' 
0 1 






























12 . 


14 . 


17 . 


22 . 


(a) 

(b) 

(c) 


2 0 
0 1 

1 0 
2 1 


3 0 
0 1 

1 4 
0 1 


; expansion in the y-direction by a factor of 3, then expansion in the a- direction by a factor of 2 
; shear in the x-direction by a factor of 4, then shear in the y-direction by a factor of 2 


0 1 
1 0 


4 0 
0 1 


1 0 
0 -2 


; expansion in the y-direction by a factor of -2, then expansion in the x-direction by a 
factor of 4, then reflection about y = x 

; shear in the x-direction by a factor of -3, then expansion in the y-direction by a 
factor of 18, then shear in the y-direction by a factor of 4 



'1 O' 

'1 O' 

'1 -3' 

(d) 

4 1 

1 18 

0 1 


(a) 


(b) 


0 1 

-5 0 


ft -1 

t\j 34 3 6-f3^3 


(a) ; 

V = T 


v = x 

(b) 



v = h 

(c) ; 

2 


v= -2x 

(d) 



'1 0 O' 

(a) 

0 0 1 


0 1 0 


'0 0 f 

(b) 

0 1 0 


1 0 0 


'0 1 O' 

(c) 

1 0 0 


0 0 1 


24. 

1 


0 

(a) *l = l : 

_0_ 

;a 2 = - 1 : 

_i_ 


'O' 


"r 

(b) A 1 = 1: 

_ 1 _ 

;a 2 = - 1 : 

_0_ 








































(c) A l = l ; 


;A 2 = -l: 


(d) 


(e) 


A = l: 


A = l: 


(f) 


(0 an odd integer multiple of 7t) A = — 1 : (1, 0), (0, 1) 
(0 an even integer multiple of n) A = 1 : (1, 0), (0, 1) 

(0 not an integer multiple of n) no real eigenvalues 


Exercise Set 9.3 (page 473) 


1 - y =- b ¥ 


3. y = 2 + 5x - 3x : 


8 . y = 4 — ,2x + 2x 2 \ if x = 12> then y = 30.4 ($30.4 thousand) 


Exercise Set 9.4 (page 479) 


Exfercise SSt-^B-^^ge-^g^ 

(a) 


1. (aMc^e^L# 


_, sin 2 j: , sin. 3 tt , smnx 

sin* | —-— I —-— | ■■■ I —-— 


3. 


(a) 

(a) ~A = 


(b) 1 
(b) 


9 3-4 

1 

2 


3 -1 


(a) Ti 


-4 — 4 

2 


1 -4 I 


5 

2 

9 

2 


2 2 

1 0 

0 -3 




A = 


fifsrn 


0 I I 

2 2 

1 0 1 

2 2 

1 1 

2 2 


0 





















(d) 


A = 


p p -Ayp 
\[2 0 0 
4/3 0 - /J 


(e) 


A = 


11 0-5 

11 0 0 

0 0-1 2 

-50 2-1 


max value = 5 at ±(1, 0); min value = — 1 at ±(0 ,1) 

(a) 

(b) max value = h /T o^ j_ f - ,= ; - z — 

2 ^ 1 / 2 O- 6 /IO /20 + 6/10 

/ i, 

, -1 1 


’ min value = 


11 — y iu 

2 


- 1 1 
/20 + 6 /To ’ 1/20-6/10 


(c) max value — 
/ 

1 


± 


= ^±j£°. at ± ( 1 - 1 — ^ ; min value = —— j 

2 ^ /20 — 6/To /20-6/10/ 2 

1 1 \ 


/To 


at 


1 _ 1 

/20 + 6 /To 7 /20-6/10 


(d) max value = —— 7 ^^- f /- 7 = > /- 7 = 

2 \ |/20 — 2/10 l /20 f - 2/10 ^ 

3 -3 \ 


> min value = —— 2^~ ^ 


_3_ -3 

/20 H 2/TO ’ /2O-2/1O 


7 . (b) 


9 . (a) 

Positive definite 

(a) 

Negative definite 

(b) 

Positive semidefinite 

(c) 

Negative semidefinite 

(d) 

Indefinite 


(e) 








































Indefinite 


(f) 


13. (c) 


16. 


(a) 


A = 


1 

n 

- 1 


- 1 


n {n — 1 ) 
- 1 


- 1 


(b) 


Exercise Set 9.6 (page 496) 


- 1 


n{n — 1 ) n{n — 1 ) 
1 - 1 


n {n — 1 ) 
- 1 


«(« — 1) «(« — 1) « (« — 1) 

Positive semidefinite 


- 1 


n{n — 1 ) 
- 1 

n in — 1 ) 

1 

n 


1. 



1 

1 


(a) 

~x\ 




>r 


*2 


1 

1 

72 




f - 



(b) 



1 

2 


■*r 




>r 


_*2_ 


2 

1 

72 








yf + 


; y\ + 


(C) 




1 

1 


■*r 




'yf 

*2 


1 

1 

72 



f2 

"f 



2 2 
^1 “^2 


(d) 




/l7-4 /l7 + 4 


"*l' 


/34-8/T7 /34+8/T7 

'71' 

*2 


1 -1 

72 



l/34-8/r7 /34+8/T7 



;(i + /n)yi+( 


2 ^ 2 - 3 ^ + 47 2 
(a) 


(b) 


-*y 


5xy 


i - /ny 2 2 


<0 













































(d) 


4x 2 - 2y 2 


(e) 


5. 


(a) 


(b) 


(c) 


(d) 


(e) 


[* y] 

[* y] 

[X y] 

[x y] 

[x y] 


2 -4 


3 

2 


1 -4 


1 

2 


4 [-7 2] 


4 [5 3] 


+ 7 = 0 


-3 = 0 


i 0 

4 0 

0 -2 


-8 = 0 


-7 = 0 


0 0 
0 1 


4 [7 -8] 


-5 = 0 


9*' 2 4 4/ 2 = 36, ellipse 

(a) 


(b) 


*' 2 - 16/ 2 = 16, hyperbola 


(c) 


y /2 — 8/, parabola 


(d) 


x* 2 + / 2 = 16, circle 


(e) 


18/ 2 - 12*' 2 = 419, hyperbola 


(f) 


/ = — parabola 


9. 2x " 2 + y " 2 = 6, ellipse 
11 * 2 * r/ 2 — 3y f>2 = 24, hyperbola 




























15. 


Two intersecting lines, y — x and y — — x 


(a) 


(b) 


No graph 


(c) 


The graph is the single point (0, 0). 


(d) 


The graph is the line y = x. 


3 | 2 _ | ru 

^ The graph consists of two parallel lines ^yy x 1 ^yyT' — -*= ^. 


(f) 


The graph is the single point (1,2). 


Exercise Set 9.7 (page 501) 


l. 


(a) 

(b) 

(c) 

(d) 


x 1 A 2 y 2 -z 2 A- Axy -5yz 


3x 2 4- 7 z 2 -I- 2 xy — 3xz A- Ayz 


xyA-xz I yz 


r? + y 2 -z 2 


(e) 


3z 2 I 3xz 


(f) 


2 z 1 + 2xz + y* 


3. 


(a) 


[ x ? z i 


2 

2 


0 -4 


0 

_ 5 
2 

- 1 




" x~ 

y 

z 

+[702] 

y 

z 


-3 = 0 


(b) 

i- 

m | r-j 

I 

m 



~x~ 

[x y 

1 0 2 

2 7 

z 

+ [ -3 0 0] 

y 

z 


2 





-4 = 0 














(C) 


[ x y z i 


o I I 

2 2 

1 0 I 

2 2 

1 I 0 

2 2 


1 = 0 


(d) [x y z] 


1 0 0 

0 1 0 

0 0-1 


-7 = 0 


(e) 

0 0 j 



- x~ 

[x y z ] 

0 0 0 
f o 3 

y 

z 

+ [0 -14 0] 

y 

z 


+ 9 



'o o r 

~x~ 


" x~ 

(f) [X y Z] 

0 1 0 

1 0 2 

y 

z 

+ [2 - 1 3] 

y 

z 


= 0 


(a) 

(b) 

(c) 


9x' 2 + 36/ 2 + 4z f 2 = 36, ellipsoid 


6x /2 4 - 3/ 2 — 2z >2 — 18, hyperboloid of one sheet 


3x >2 — 3y f2 —z f 2 = 3, hyperboloid of two sheets 


(d) 


4x’ 2 -f 9y /2 — z /2 = 0, elliptic cone 


(e) 


x' 2 4-16./ 2 - 16z' = 0, elliptic paraboloid 


(f) 


lx’ 2 — 3y’ 2 -\-z ! = 0, hyperbolic paraboloid 


(g) 


x t2 + y* 2 + z' 2 = 25, sphere 


9. 2 z ,/2 = —1, hyperboloid of two sheets 

11* x l>2 —y r * 2 +z fl = 0, hyperbolic paraboloid 

Exercise Set 9.8 (page 509) 


Multiplications: mpn; additions: mp{n — 1) 






















3 . 


Solve Ax — b by Gauss-Jordan elimination 


Solve Ax — b by Gaussian elimination 


Find ^4 _1 by reducing [j 4|/] to [l 


^ _1 ] 


Solve Ax = b&s x = A~ l b 


Find det(j4) by row reduction 


Solve Ax — b by Cramer's Rule 


n — 5 

+: 50 
x: 65 
+: 50 
x: 65 
+: 80 
x: 125 
+: 100 
x: 150 
+: 30 
x: 44 
+: 180 

x: 264 


« = 10 


+: 375 
x: 430 
+: 375 
x: 430 
+: 810 
x:1000 
+: 900 
x: 1100 
+: 285 
x: 339 
+: 3135 

x: 3729 


« = 100 


+: 383,250 
x: 343,300 
+: 383,250 
x: 343,300 
+: 980,100 
x: 1,000,000 
+: 990,000 
x: 1,010,000 
+: 328,350 
x: 333,399 
+: 33,163,350 

x: 33,673,399 


» = 1000 

+: 333,283,500 
x: 334,333,000 
+: 333,283,500 
x: 334,333,000 
+: 998,001,000 
x: 1,000,000,000 
+: 999,000,000 
x: 1,001,000,000 
+: 332,833,500 
x: 333,333,999 
+: 33,316,633 x 10 4 
x: 33,366,733 x 10 4 




* = 5 

n = 10 

II 

O 

o 

« = 1000 



Execution Time 

Execution Time 

Execution Time 

Execution Time 



(sec) 

(sec) 

(sec) 

(sec) 

Solve = 1) by 

Gauss-Jordan elimination 

1.55 xlO -4 

1.05 xlO -3 

.878 

836 

Solve j4x = b by Gaussian 
elimination 

1.55 x 10 -4 

1.05 x 10 -3 

.878 

836 

Find ^-1 by reducing [j 4|/] to 

2.84 x 10 _4 

2.41 x 10 -3 

2.49 

2499 

[/ 

^ _1 ] 





Solve 2lx = 1) as x = 2l"~ 1 b 

3.50 x 10 _4 

2.65 x 10 -3 

2.52 

2502 

Find det(j4) by row reduction 

1.03 x 10 -4 

3.21 x 10 -4 

.831 

833 
























m = 5 

« = 10 

II 

O 

o 

« = 1000 


Execution Time 
(sec) 

Execution Time 

(sec) 

Execution Time 
(sec) 

Execution Time 
(sec) 

Solve Ax = h by Cramer's 
Rule 

6 . IS x 10 -4 

90.3 x 10 -4 

83.9 

834 x10 3 


Exercise Set 9.9 (page 517) 

1 * = 2> x 2 = 1 

3. *1 = 3> x 2 = — 1 

x\ = — 1 > x 2 = 1 ’ *3 = 0 

*1 = — 1> X 2 — 1’ *3 = 0 


9. 


x\ = — 3> X 2 = 1’ *3 = 2> X 4 = 1 


11 . 


(a) 


A = LU = 


2 0 0 

-2 1 0 

2 1 1 


0 0 
0 0 


1 

2 

1 

0 


(b) 

”10 0' 

1- 

o 

o 

CM 

1 

A = L\DU = 

-1 1 0 

0 1 0 

o 


1 1 1 

0 0 1 

0 


10 0" 

'2 1 - ' 

f 

(c) a = l 2 u 2 = 

-1 1 0 

0 0 

1 


1 1 1 

0 0 0 


1 

2 

1 

0 


13. 

a b 


' 1 O' 

a b 

(b) 

c d 


1 

a 

0 ad — be 
a 


18. 


A —FLU — 


1 0 0 
0 0 1 
0 1 0 


3 0 0 
0 2 0 
3 0 1 



0 


1 

0 


1 

2 

1 


0 



































Exercise Set 10.1 (page 526) 


1 . (a-d) 



x = — 2 ’ y = —3 
(a) 


(b) 


x = 2, y = 1 


2 + 3 i 

(a) 

— l — 2i 

(b) 

-2-1-93 

(c) 













( c) ziz2 = -j--i’*i =j(-3 + 4i)’Z2 = 
11. 76 — 88: 


12. 26 — 18; 


(2 + /2) + i(l-/2) 


18. — 24j 


20 . 


(a) 


13 + 13; — 3 H 12: — 33 — 22s 
1 +: 0 : 

7 + 9: —6-1-6: —16 — 16: 


(b) 


6 + 2: -11+19: 

- 1 + 6: -9-5: 


(c) 


6 : 1 4 : 
-6-i 5-9: 


(d) 


22—7: 2-1-10: 
-5-4: 6-8: 

9 —: — 1 —: 


z= — 1 ±: 
(a) 



Exercise Set 10.2 (page 531) 


2-7: 

(a) 

-8 + 5 : 

(b) 


-5: 

(c) 

(d) 


l 










-9 


(e) 


0 


(f) 


5. 


— ! 


(b) 



li 

(c) 


9. 

625 625 


U. 1-/3 + 1 + /3 . 


-1 -2i 

(a) 

(b) 25 25 


a.v 



(b) 



(c) 










*'s' 

i 


X 


(d) 



t y 

X 



-i 

L 


*1*2 I Z1Z2 

(a) ^ +y j 

*2x1 -*172 

0 » ,j + yj 


27. 


(c) 


Yes, if zi* 0- 


30. 1 , . 0 1 

*1 = 2 + ! ’ *2 = 2 > *3 = 2 


i — (1 -h z)i» *2 — 2^ 


35. 


(a) 

(b) 


z 2 

- 1 z 

0 1 

- i 2 i 


39. 


(a) 


(b) 


— i — 2 — 2z — 1 + z 

1 2 — z 

i i 1 

1 + z — z 1 

— 7 -I- 6z 5 — z 1-1-4z 

1 -I- 2z — z 1 


Exercise Set 10.3 (page 539) 


o 

(a) 

















7T / 2 

(b) 

-W2 

(c) 

7T / 4 

(d) 

2W3 

(e) 

— 7T / 4 

(f) 


2|" cos ("o 

(a) L 


+ ,S ”'2iJ 


4 [cos « -1- i sin ~] 


(b) 




+ i sin i t 


(C) ■ L WJ 14 


(d) 


12 


COS[-y-j -I- ! sin^y- 


(e) 


3^2 


cosf -^) + !sm(--^ 


(f) 


4 cos (-f) +is ”(--?) 



(b) 













The roots are ± { 2 }^ 2^^^ ± ^2 1/4 


2 1 /4 i) and the factorization is 















z 4 4- 8 = (z 2 - 2 J/4 z4- 2 3/2 )(z 2 4- 2 5/4 z+ 2 3/2 ). 


Re (z) = - 3, Irn(z) = 0 

(a) 

Re(z) = — 3> Im(z) = 0 

(b) 

Re(z) = 0, Irn(z) = - ^2 

(c) 


Re(z) = — 3> Im(z) = 0 

(d) 


20. cos 25 = cos 2 5 — sin 2 5> sin25 = 2 sin0cos5> 

cos 35 = cos 3 5 — 3 sin 2 5 cos 5> sin 35 = 3 sin 5 cos 2 5 — sin 3 5 


Exercise Set 10.4 (page 544) 


(3i, -i, -2- h 4) 
(a) 


(3 + 2i, —1 — 2 i, — 3 4- 5j, — i) 

(b) 


(- 1 -2i, 2i,2-i, - 1) 
(c) 


(-3 + 9i, 3-3i, - 3-Si, 12 f 3i) 

(d) 


( - 3 h 2i, 3, - 3 - 3i, i) 
(e) 


( — 1 — 5i, 3i, 4, -5) 

(f) 


J2 

(a) 





9. 


3 


(a) 

2 — 27s 

(b) 

-5-10j 

(c) 


Not a vector space. Axiom 6 fails; that is, the set is not closed under scalar multiplication. (Multiply by i, for 
example.) 


13. 

leer T is all multiples of 


1 4 3 i 
1 4 ! 
— 2 


; nullity of T — 1 


( — 3 — 2!)u 4 (3 — i)v 4 (1 4 2 j)w 
(a) 


(2 f i)u 4- ( — 1 4 i) v 4- ( - 1 — j) w 

(b) 

Ou 4 - Ov 4 Oiv 

(c) 

( — 5 — 4i)u 4 (5 4 2i)v 4(2 4 4i)w 

(d) . 


19. (a), (b), (c) 

21 . (b), (c) 

23. f — 3g — 31i = 0 

25. (a), (b) 

27. ( — 1 — i, 1); dimension = 1 

— 2 ’ ^ 4 ^’ 4 *’ ^dimension = 2 


Exercise Set 10.5 (page 551) 




2. 


-12 


(a) 

0 

(b) 

2 i 

(c) 

37 

(d) 


-4 + 5i 

(a) 

0 

(b) 

4 — 4z 

(c) 

42 

(d) 


6. — 9 — 5i 


8 No. Axiom 4 fails. 


10 . 


(a) 


Jio 


2 

(b) 



(d) 


0 




14 . 


2 


(a) 



16 . 


(a) 

(b) 


71/2 
2/3 


20. (b) 


23 . 


-L_ n 0 — 

\fifi 


; n 2; i ^ / 2; 3; 2; — 2; 

, ,/ 6 ’ ’ ^ 6 ’ /ej’ [/2T’ /2T’ /2T’ /2T J’ 


; 2 ; 


25 . 


(a) V 1 = -7=. “Tr. “ 7 = > v 2 — 


^3’ 


(b) V! - (;, 0, 0 ), V 2 = p, 


-^k “7=. 0 1, V3 = 


, 


^ 2 ’ /2 


Z Z 


2i 


V 


li — 2 i 
^53’ |/53 


.v 3 =P, 


fi' |/6' ^ 

2; 7; \ 

fi?' 


27 . 


▼1 = 


n i 1 —! 1 „ _ r 3 i 2 1 -I- j 

U,- 7 ^,— —\,\2=' -- - 


fi’ fi)’ 


/l5’ /l5’ /l5 


36 . r=. , 3 1 

"“-^vr-vr 3 


Exercise Set 10.6 (page 561) 


1 . 


(a) 


— 2i 4 5-i 

1 -I- i 3 —i 0 


(b) 


(c) 


(d) 


— 2 i 4 —i 

1 + i 5+ li 3 
— 1 — i i 1 

' -li 
0 
3 i 

a n a 2 1 
^12 G 22 
a 13 «23 


,/?■ fi) 
























3 . k = 3 + 5i, l — i, m — 2 — Ai 


4 . (a), (b) 


5 . 


(a) 

(b) 


(c) 



2/2 

1 

2/2 


(1 - |/3,) 
(-fl-i) 


(d) 




1 —; _ ; 3 —; 

2 p 2/l5 

_I JL 4-3; 

2 /3 2^15 

J_ ; _ 5; 

2 p 2/l5 


7 . 


P = 


- 1 


1 

P 


1 —z 


/6 

_ 2 _ 

/6 


? 


3 0 
0 6 


9 . 


f = 


1 4 ; 


fe 

_ 2 _ 

f6 


1 4 i 



■P~ 1 AP = 

? 


2 0 
0 8 


11 . 


P = 


0 1 0 


1 —; 



1 — ; 


P 


■P~ l 2 AP = 

? 


1 0 
0 5 
0 0 


0 

0 

-2 






































14 . 


(a) 


0 i 
- i 0 


is one possibility. 


Supplementary Exercises (page 563) 


3. 

— i 


"l" 


1 


0 


0 


l 


is one possibility. 


5. A= 1, w, uj 2 ( = uj) 

Exercise Set 11.1 (page 572) 

y = 3x — 4 

(a) 

y= 

(b) 


x 2 f y 2 -4x-6y + 4 = 0 or (. x-2) 2 4- (y - 3) 2 = 9 

(a) 

x 2 4-y 2 4- 2x-4y-20 = 0 or (x 4- l) 2 4- (y - 2) 2 = 25 

(b) 


3. x 2 4- 2xy 4-y 2 — 2x 4- y — 0 (a parabola) 


x 4-2y | z = 0 

(a) 


— x 4- y — 2z fl = 0 

(b) 


5 . 


(a) 


x y z 
*1 ^1 ^1 
*2 J >2 22 
x 3 y 3 23 


o 

l 

l 

l 


x 4- 2y 4- z — O', — x 4- y — 2z—0 

(b) 










6 . 


x 2 4- y 2 4- z 2 — 2x — 4y — 2z — — 2 or (jr — 1) 2 | (y — 2) 2 4- (z — l) 2 = 4 

(a) 


x 2 + y 2 4 z 2 - 2x - 2y == 3 or (x - l) 2 4 O - l) 2 4 z 2 — 5 

(b) 


10 . 


y x 2 x 1 

71 *i *i 1 

72 *2 *2 1 

73 *3 1 


The equation of the line through the three collinear points 


12 . 0 = 0 


13 . The equation of the plane through the four coplanar points 


Exercise Set 11.2 (page 576) 


1. , _ 251 , __97_ , 158 

1 317 ’ 12 317 ’ 13 317 


2. 7 13 7 2 7 11 

u = —’’ h = - ^3 = — 


3 . 7 _ _5_ j _J_ , __6_ 

7l 22 ’ 12 22’ /3 11 


/l = i, l 2 = 0, / 3 = 0, / 4 = ^ / 5 = i,/ 6 = i 


Exercise Set 11.3 (page 588) 


x\ = 2> X 2 = —’ maximum value of z = 


No feasible solutions 


3 Unbounded solution 







Invest $6000 in bond A and $4000 in bond B; the annual yield is $880. 


c 7 77 

j- cup of milk, =^r ounces of com flakes; minimum cost = - lLLL 


18 


18 


= 18.6 $ 


x\ > ij and X 2 > 0 are nonbinding; 2xi + 3*2 < 24 is binding 
(a) 


x i — X 2 < v for v <■ — 3 is binding and for y < — 6 yields the empty set. 

(b) 


^2 < v for v < 8 is binding and for v < 0 yields the empty set. 
(c) 


550 containers from company A and 300 containers from company B\ maximum shipping charges = $2110 
925 containers from company A and no containers from company B\ maximum shipping charges = $2312.50 
0.4 pound of ingredient A and 2.4 pounds of ingredient B; minimum cost = 24.8 <t 


Exercise Set 11.4 (page 595) 

1. 700 

2 . 5 

(a) 

4 

(b) 


4 . 


(a) 


Ox, units; sheep, -|y- units 


Q .7 4 

First kind, — measures; second kind, — measures; third kind, — measures 
(b) 25 25 25 


(a) * 

(b) 


(a 2 I 33 I - I . *j =a,- — *i,; = 2. 3, ... ,n 

n — 2 

Exercise 7(b); gold, 30-^-minae; brass, 9-^-minae; tin, 14-^-minae; iron, 5-^-minae 




5x+y-\-z — K = 0 

(a) x + ly + z -K = § 

x \ y + < &z-K=§ 


x — ’ y — 2 ~ |^i ’ K = £ w herc t is an arbitrary number 

Take t = 131, so that x = 21,y = 14, z = 12, K= 131- 

(b) 


Take t = 262, so that * = 42, y = 23, z = 24, J6T = 262- 
(c) 


7 ? 

Legitimate son, 577^- staters; illegitimate son, 422-^ staters 

(a) y y 


Gold, 30-^- minae; brass, 94- minae; tin, 14-^- minae; iron, 5-^ minae 
(b) 2 2 2 2 


First person, 45; second person, 37-^; third person, 22^- 
(c) 2 2 


Exercise Set 11.5 (page 606) 


S(x) = — .12643(3: — ,4) 3 — ,20211(x — ,4) 2 + ,92158(x — .4) + .38942 
(a) 


S(.5) — .47943; error = 0% 

(b) 


The cubic runout spline 
(a) 


S(x) = 3x 2 -2x 2 +5x+\ 

(b) 


4. 

1 -,00000042(x4- 10) 3 

+ .000214(x + 10) I 

.99815, 

- 10 < x < 

SCO = < 

.00000024(x) 3 

,0000126(x) 2 + .000088(x) 1 

.99987, 

0 < x < 





— . 00000004 (j: — 10) 3 — 

,0000054(x- 10) 2 -,000092(x- 10) 1 

-.99973, 

10 < x < 


,00000022(x — 20) 3 — 

V 

. 0000066 (tt — 20) 2 — ,000212(x — 20) 1 

.99823, 

20 < x < 


0 

10 

20 

30 


Maximum at (*, S(x)) = (3.93, 1.00004) 







5 . 




£(*) = 




.0000009(^-1 10) 3 — .0000121(x -h 10) 2 H ,000282(x +10)+.99815, 

,0000009(x) 3 — . 0000093 (jt) 2 + .000070(x) +.99987, 

.0000004(x - 10) 3 - ,0000066(x - 10) 2 - ,000087(x - 10) + .99973, 

.0000004(x - 20) 3 - ,0000053(x - 20) 2 - ,000207(x - 20) + .99823, 


Maximum at (*, £(x)) = (4.00, 1.00001) 


6 . 


(a) 


S(x) = 


— 4x 3 + 3x 0 < x < 0.5 

4x 3 - 12x 2 + 9x- 1 0.5 <x < 1 


(b) S & = 


2 — 2x 
2 — 2x 


0.5<x< 1 
1 <x< 1.5 


(c) 


The three data points are collinear. 


(b) 

'4 

1 

0 

0 - 

■■ 0 

0 

0 

l" 

M i 


y«-i - 

■ 2yi + 

72 

1 

4 

1 

0 ■■ 

■■ 0 

0 

0 

0 

m 2 


y i - 

■ 2y 2 + 

73 


0 

1 

4 

1 - 

■■ 0 

0 

0 

0 

m 3 

6 

72“ 

■ 2y 3 + 

74 


: 

: 

: 

: 


: 

■ 

: 


" h 2 





0 

0 

0 

o ■ 

■■ 0 

i 

4 

i 

M h _ 2 


y»- 3- 

' 2 —1 


i 

0 

0 

0 - 

■■ 0 

0 

1 

4 

M-n—i 


i 

a 

1 

to 

1 

■ 27h-i + 

71 

(b) 

'2 

l 

0 

0 - 

- 0 

0 

0 

l" 

" Mi 


1 

- 71 + 

72 

1 

4 

1 

0 ■ 

■■ 0 

0 

0 

0 

m 2 


71 - 

- 272 + 

73 


0 

1 

4 

1 - 

■■ 0 

0 

0 

0 

m 3 

6 

72- 

- 273 + 

74 


: 

■ 

: 

: 


■ 

: 

: 

: 

~h 2 


: 



0 

0 

0 

o - 

■■ 0 

0 

4 

i 

Myi — 1 


7m —2 - 

- 27h-1 + 

7h 


0 

0 

0 

0 ■■ 

■■ 0 

1 

1 

2 

M n 


7n-l - 

7«H ^7m 


Exercise Set 11.6 (page 617) 


1 . 

x (!) - 

/ \ JL - 

'.4" 

x® = 

'.46" 

x® = 

".454" 

x«> = 

".4546" 

x^ = 

'.45454' 


(a) 

.6 


.54 


.546 


.5454 


.54546 


(b) 


P is regular, since all entries of P are positive; q = 


_1 

11 

_ 6 _ 

11 


(a) xC 1 ) = 


'.7' 

x© = 

'.23' 

5 

".273" 

.2 

.52 

.396 

.1 


.25 


.331 


10 < x < 0 
0 <x< 10 
10 <x < 20 
20 < x < 30 


2 . 


































(b) 


P is regular, since all entries of P are positive: q = 


22 

72 

29 

72 

21 

72 


3 . 


(a) 


(b) 


(c) 


_9_ 

17 

_ 8 _ 

17 

26 

45 

11 

45 

J_ 

19 

_4_ 

19 

11 

19 


4 . 



(b) 


pn 



, n = 1, 2, .... Thus, no integer power of P has all positive entries. 


0 

1 


0 

1 


as n increases. 


so 


P» X (QD 


0 

1 


for any as n increases. 


(c) 


The entries of the limiting vector 


are not all positive. 


' 1 1 r 


' 1" 

2 4 4 


3 

1 1 1 

4 2 4 

has all positive entries; q = 

1 

3 

1 1 1 


1 

4 4 2 


3 


13 

54 - 7 % in region 1, 16 - 7 % in region 2, and 2 9- 7 % in region 3 
6 3 6 


Exercise Set 11.7 (page 627) 






















1 


(a) 


0 0 0 1 
10 11 
110 1 
0 0 0 0 


(b) 


0 110 0 
0 0 0 0 1 
10 0 10 
0 0 10 0 
0 0 10 0 


(c) 


0 10 10 0 
1 0 0 0 0 0 
0 10 111 
0 0 0 0 0 1 
0 0 0 0 0 1 
0 0 10 10 
























(b) 


1 -step: P\ —► P2 

2 -step: P\ —> P4 —► P2 

Pi P 3 ^ P 2 

3 -step: P\ —► P'i —► P\ —> P2 


(c) 


^1 

—> 

P 3 

—> 

Pa 

—» 

P2 

Pi 


Pa 


P3 


P2 

1- 

step: 


Pi- 


Pa 


2-step 


Pi - 


P3- 


Pa 

■step: 

Pl- 


P2- 


P\- 


Pi 

—> 

Pa 

—„ 

P3 

—> 

Pa 


Pa 


4 . 


(a) 


(c) 


1 0 0 0 0 

0 10 0 0 

0 0 110 
0 0 12 1 

0 0 0 1 2 

The i yth entry is the number of family members who influence both the jth and yth family members. 


(Pl,P2,P3) 

(a) 


(b) 


{P2.PA.P5) 


(c) 


{.P2.P4.Pt.Pt) 1111(1 {Pi. Pi, Pi) 


None 

(a) 


(b) 


{Pi,Pa,P6} 


7 . r 


0 

0 

1 

f 

Power of Pi = 5 

1 

0 

0 

0 

Power ofP2 — 3 

0 

1 

0 

1 

Power of^3 = 4 

0 

1 

0 

0 

Power ofP4 = 2 


8 First, A; second, B and E (tie); fourth, C; fifth, D 


Exercise Set 11.8 (page 637) 






1 . 


-5/8 


(a) 


(b) 


[0 1 0 ] 


(c) 


[ 1000 ] 


2 . 


Let j4 = 


1 1 
1 1 


, for example. 


3 . 


(a) P = [0 1]. 1 - 


»v = 3 


(b) p = [o l o ], q 


> v = 2 


(c) p *= [o 0 1 ], 1 * = 


>v = 2 


(d) p* = [0 1 0 0], <1* = 


5 v = — 2 


(a) * 

P 


(b) * 

P 


5 2 
8 8 


2 1 
3 3 


. 1 


* 

»'1 = 


, v = 


27 


, v = ■ 


70 


(c) P = [1 0M = 


i v = 3 


(d> . 


(e) * 

P 


3 2 
5 5 


q = 


, V — 


19 


3_ 10 
13 13 


»'1 = 


13 

12 

13 


, v = 


-29 

13 


5 . 


* 

1 > = 


12 _Z_ 

20 20 


'1 = 


li 

20 

_9_ 

20 


, v = — 


3_ 

20 



































Exercise Set 11.9 (page 646) 


(a) 

_3_ 


"6" 

(b) 

5 


6 


'78 

(c) 

54 


79 


Use Corollary 11.9.4; all row sums are less than one. 

(a) 

Use Corollary 11.9.5; all column sums are less than one. 

(b) 


(c) Use Theorem 11.9.3, with x: = 

'2' 

1 

>Cx = 

'1.9' 

.9 


1 


.9 


E 2 has all positive entries. 

Price of tomatoes, $120.00; price of corn, $100.00; price of lettuce, $106.67 

5. $1256 for the CE, $1448 for the EE, $1556 for the ME 

6. 542. 

(b) 503 

Exercise Set 11.10 (page 655) 

The second class; $15,000 

2. $223 

3. 1:1.90:3.02:4.24:5.00 

si (gr 1 +S2 -1 +“‘ + ejrii) 













6 . 1:2:3:--:*-1 


Exercise Set 11.11 (page 662) 


l. 


(a) 


0 110 
0 0 11 
0 0 0 0 


(b) 



(c) 



(d) 



.500 

.366 

0 


(b) (0,0,0), (1,0,0), (li 1, 0\ and 



1 0 0 
(a) 0-10 

0 0 1 















































(b) 


-10 0 
0 1 0 

0 0 1 







> 

1 \ 



ft 








(c) 


'1 0 

0 1 

0 0- 

o' 

0 

1 

~ f> 


II 

n 


4 . 


(a) 


Mi = 


M 4 = 


2 ° ° 
0 2 0 

0 0 ^ 


,M 2 = 


1 1 

2 2 

0 - 0 

0 - 0 


,M 3 = 


0 

cos 20° 
sin20° 


cos( — 45°) 

0 

sin( — 45°) ' 


'0 

- 1 

O' 

0 

1 

0 

,m 5 = 

1 

0 

0 

— sin( — 45°) 

0 

cos( -45°) 


0 

0 

1 


P' = M 5 M 4 M 3 (MiP-\ m 2 ) 
(b) 


0 

— sin 20° , 
cos 20° 


3 

0 

O' 


'1 

0 

0 


'1 

1 ■ 

■■ r 


' cos 35° 

0 

sin 35°' 

0 

.5 

0 

, m 2 = 

0 

cos 45° 

— sin45° 

II 

0"l 

0 

0 ■■ 

■■ 0 

II 

’rr 

0 

1 

0 

0 

0 

1 


0 

sin 45° 

cos 45° 


0 

0 ■' 

'■ 0 


-sin 35° 

0 

cos 35° 



cos( — 45°) — sin( — 45°) 0 


'0 0 ■ 

■■ o' 


1 

[■■O 

o 

o 

m 5 = 

sin( — 45°) cos( — 45°) 0 

0 0 1 

,m 6 = 

0 0 ■ 

1 1 ■ 

■■ 0 

.. 1 

,m 7 = 

0 1 0 

0 0 1 


P ! = M 7 {M 5 M 4 {M 2 M { P hM 3 )-\ M 6 ) 


(b) 



1 

n 

o 

Oj 

Vj 


0 

sin 3 


cos a 

— sinct 

o' 


cos 0 

0 

sin# 


cos a: 

sina 

o' 

*1 = 

0 


1 

0 

, r 2 = 

sin a 1 

cos Q 1 

0 

.*3 = 

0 

1 

0 

9 R4 = 

— sin a- 

COS G 

0 , 


— sin. 

3 

0 

cos 3 


0 

0 

1 


— sin# 

0 

cos 9 


0 

0 

1 


cos 3 

0 


— sin 3 










r 5 = 

0 

1 


0 











sin 3 

0 


cos t 3 



































































(a) 


M = 


10 0 
0 1 0 70 
0 0 1 
0 0 0 


(b) 


ZO 

1 


10 0-5 
0 10 9 

0 0 1-3 
0 0 0 1 


Exercise Set 11.12 (page 673) 


l. 


(a) 


t\ 
L2 

£■3 

t A 


(b) 


t = 


(c) 


t O) = 


1 1 
4 4 


0 4 4 0 


4 0 0 4 


4 0 0 4 


1 i 

4 4 


0 4 4 0 




'o’ 

pi" 


1 

£2 


2 

£3 

+ 

0 

p4 


1 



2 







’ 1 ’ 


’ 3 ’ 


’ 7 ’ 


’ 15 ’ 


1 ’ 

o' 


8 


16 


32 


64 


64 

1 


5 


11 


23 


47 


1 

2 

t <2) _ 

5 1 

8 

,t® = 

16 

,t^ = 

32 

,t® = 

64 


64 

0 

1 

3 

7 

15 


1 

1 


3 


16 


32 


64 


64 

2 


5 


11 


23 


47 


1 



3 


16 


32 


64 


64 


(d) 


for^j, 4.5%; for ^ ~ 1-8% 


2 . 1 

2 


t 0) = 
tC0 = 


r 3 

5 2 

5 

4 

2 

5 

4 

3] 

T 



4 

4 4 

4 

4 

4 

4 

4 

4 _ 




' 13 

18 

9 

22 


13 

7 

21 

16 

10 ' 

. 16 

16 

16 

16 


16 

16 

16 

16 

16 _ 


































Exercise Set 11.13 (page 685) 


* (3\_ 27) 

(c) 3 1,22’22 J 

(a) x f = (1.40000, 1.20000) 
xf = (1.41000, 1.23000) 
xf = (1.40900, 1.22700) 
xf = (1.40910, 1.22730) 
xf = (1.40909, 1.22727) 

xf = (1.40909, 1.22727) 

Same as part (a) 

(b) 

(c) xf = (9.55000, 25.65000) 
xf = (.59500, - 1.21500) 
xf = (1.49050, 1.47150) 
xf = (1.40095, 1.20285) 
xf = (1.40991, 1.22972) 
xf = (1.40901, 1.22703) 

4 * i* = (l, l)«x 2 * = (2,0)>x 3 * = (1, 1) 

X 1 + 4- x 9 = 13.00 

X 4 + x$ 4 - x^ = 15.00 
xi -I- x 2 4 X 3 = 8.00 
. 82843 (xg 4 - x 8 ) + .58579x9 = 14.79 
1.41421(x 3 + xj + x 7 ) = 14.31 
,82843(x 2 + x 4 ) 4- .58579x1 = 3.81 
x 3 + xg + X 9 = 18.00 
x 2 + x^ 4 -xg = 12.00 
xi 4- X 4 4- X 7 = 6.00 
.82843(x 2 4- x 6 ) 4 .58579x 3 = 10.51 
1.41421 (xi +x 5 + x 9 ) = 16.13 
,82843(x 4 +xg) 4 .58579x7= 7.04 


*7 T 4- *9 = 13.00 
*4 4 *5 4 *6 = 15.00 
*1 -I- *2 T- *3 = 8.00 
,04289(* 3 + x s 4- x 7 ) 4 ,75000(* 6 4 *g) 4 .61396*9 = 14.79 
.91421 (*3 4 *3 4 *7) 4 .25000(*2 T *4 4 *6 + *g) = 14.31 
,04289(*3 4 *5 4 *7) 4 .75000(*2 4 *4) 4 .61396*1 = 3.81 

*3 4 *g 4 *9 — 18.00 

*2 - I - *5 + = 12.00 

*1 4 *4 4 *7 = 6.00 
.04289(*1 4 *5 4 *9) 4 .75000(* 2 4 x 6 ) 4 .61396* 3 = 10.51 
.91421 (*1 4 *5 4 *9) 4 .25000(*2 4 *4 4 x $ 4 *g) = 16.13 
,04289(*i 4 *5 4 *9) 4 .75000(*4 4 *g) 4 .61396*7 = 7.04 


Exercise Set 11.14 (page 702) 


1 . 


Ti 


([*1 

)-12 

'1 O' 

"*" 

1 

"*1 ' 

IH 

) 25 

0 1_ 

y 

~r 

ft 


1 = 


1, 2, 3, 4, where the four values of 










’ 13 ’ 



'o' 


' 13 ' 


' 0 ' 


25 


are 



25 


13 

, and 


ft 


_ 0 _ 


0 


25 


13 

25 


,*H<^=]n(4)/ln(||) = 1.888... 


s pc .47; d H (Z) Pd In(4) / ln(l / .47) = 1.8.Rotation angles: 0° (upper left); 90° (upper right); 180° (lower left); 

180° (lower right) 


3. 



all rotation angles are 0°; 

ii. 


d H (S) — In(7) /ln(3) = 1.771 .... 

iii. 


This set is a fractal. 



all rotation angles are 180°; 

ii. 


d H (S) — ln(3) /ln(2) = 1.584 .... 

iii. 


This set is a fractal. 




















(C) 


1 . 


s 


1 . 

2 ’ 


rotation angles: 90° (top); 180° (lower left); 180° (lower right); 

ii. 


d H (Z) = ln(3) /ln(2) = 1.584 .... 

iii. 


This set is a fractal. 



rotation angles: 90° (upper left); 180° (upper right); 180°(lower right); 

ii. 

d H (S) = ln(3) /ln(2) = 1.584 .... 

iii. 

This set is a fractal. 


4. s = .8509..., 9 = -2.69°.. 

5 (0.766, 0.996) rounded to three decimal places 
6. d^fs) = ln( 16) /ln(4) = 2 
7 - ln(4) / lnfy'j = 4.818... 


djj(S) = ln(8) / b(2) = 3; the cube is not a fractal. 
k = 20; s = y; dn(S) = ln(20) / b(3) = 2.726 ....; the set is a fractal 


10 . 


Stcojui nemie 


Thirtl Ik'l.ik 


. . " + ■■ ■" " - k Fciurth ilmte 

dft f S) = ln(2)/to{3) = 0,6309 ♦ .* 


11 . 


Area of £ 0 = l; area of £j = |. = 0.888. 
/Q \4 

S 4 = f = 0.624... 


area of 


8 


2 

= 0.790...; area of £3 



= 0.702...; area of 


Exercise Set 11.15 (page 716) 


1. 11(250) =750, n(25) = 50, n(125) = 250, n(30) = 60, n(10) = 30, n(50) = 150, 11(3750) =7500, 
n(6) = 12 , n(5) = 10 


One 1-cycle: {(0, 0)} : one 3-cycle: joY [|, |\ fo, |] j.; 


,wo 4-cycles: < (£. o), [± |\ (f, o). (§, §)} and {(o, §), (§, f), (o. f), (£. f) 


,wo 12-cycles: {(, 1). (f §), (f. f} (|, *). (f f } (*. f). (o. f ). (f . f } (f (f f ). (f . f) (f . * 

- {(?■»} (i- i) (I- !)■ (I ■ I)- (i- !)■ (f i). (!■ »)■ (?■ f )■ (!■ !)■ ({■ I} (f ■ §)■ (I ■! 

n(6) = 12 


3,7, 10, 2, 12, 14, 11, 10, 6, 1, 7, 8, 0, 8, 8, 1, 9, 10, 4, 14, 3, 2, 5, 7, 12, 4, 1, 5, 6, 11, 2, 13, 0, 13, 13, 11, 9, 5, 
(a) 14,4,3,7,... 


(c) 


(5,5), (10,15), (4,19), (2,0), (2,2), (4,6), (10,16), (5,0), (5,5),... 


4. 


(c) 1,16 flist flve itera,es of (lor °) are (lor Tor)- (lor lor)' (lor lor)' 
f J4- J5.Y 
Uoi ’ ioi J 


13 21 

101 ’ 101 


I, and 

















6. 


^ The matrices of Anosov automorphisms are 


3 2 

1 1 


and 


5 7 
2 3 


(c) 


The transformation affects a rotation of S through 90° in the clockwise direction. 


9. (0, l> 


(l. It 



11 , i/’t 


IB] 



<0. (I) (1/2.01 (1.0) 


In region I: 


~a~ 


'O' 

h_ 


_0_ 


; in region II: 




O' 

h_ 


_ - 1_ 


; in region III: 


a 


- 1 
- 1 


; in region IV 


yj and y j form one 2-cycle, and y j and ^ j form another 2-cycle. 


Exercise Set 11.16 (page 729) 


GIYUOKEVBH 

(a) 


SFANEFZWJH 


(b) 


2 . 


(a) 


A~ l = 


12 7 

23 15 


Not invertible 


(b) 


(c) 


A 1 = 


1 19 
23 24 


Not invertible 


(d) 


Not invertible 


(e) 


(f) 


A~ l = 


15 12 
21 5 


3 WE LOVE MATH 






































Deciphering matrix = 


7 15 
6 5 


; enciphering matrix — 


7 5 
2 15 


5. THEY SPLIT THE ATOM 


6. I HAVE COME TO BURY CAESAR 


7. 


010100001 

(a) 



'0 1 f 

(b) 

1 1 1 


1 0 1 


A is invertible modulo 29 if and only if det(24) * 0(mod 29)- 

Exercise Set 11.17 (page 741) 


1 /r H+1 

= 4 + 2 


(ao-co) 


byi ~ 2 
Cy! = 4 ~ 


) n = 1, 2, 


M+l 


(«0-^o) 




1 

4 

1 

4 


3. 


2 1 ' 

< 32 h +1 = 7 + ‘TTTTTr^O -^0 -^ o ) 

J 6(4) 

* 2 m +1 = i “ 777771 ( 2a 0 - *0 - 4 c o ) / " = 2 ' 


3 6(4)’ 


^2h+1 = 0 


^2h = ^ + 6(4) » _ ^0 - 4 c 0 ) 

= 2 

C2h = U ~ 6(4) M (2 ^ 0 “ “ 4C0) 


\ « = 1, 2, 


4. 


Eigenvalues: Aj = 1, A 2 = —» eigenvectors: ej = 


> e 2 = 


5 12 generations; .006% 
























6 . 


xW = 


I + . 


j2h+3 


{-3-i/ 5)(1 + /5)'’ +, + (-34 /5)(l-/5) 

. M Hh 1 


\2h+ 1 


hH-1 


1 


1 

1 


1 

if 


>+^)" +, + 

(1-/5) 

>+^)"+( 1 


>+^)"+( 1 


'( 1 + /5)" +1 + 

(!-/5) 


>2 mH-3 


m Hh 1 

mH-1 


(_3_/5)(l4/5)” +(-3+/5)(l-/5) 


mH-1 


; x<"> 


as n 


8 . 


10 0 0 
0 0 0 0 
0 0 0 0 
0 0 0 1 


Exercise Set 11.18 (page 751) 
l. 


3 v , _ 


(a > Ai = |,X! 


(b) 


(c) 




100 

50 


v©- 


175 

50 




250 

83 




382 

125 


vG> = 


570 

191 


^ = Zx© = 


857 

285 


, ^ Ajx^ = 


855 

287 


7. 2.375 


8. 1.49611 


Exercise Set 11.19 (page 760) 


1. 


(a) 


Yield = 33y% of population; xj 


1 

1 

3 

J_ 

18 


DO 









































(b) 


Yield = 45.8% of population; Xl = 


; harvest 57.9% of youngest age class 


2 . 


xi 


1.000 


2.090 

.845 


.845 

.824 


.824 

.795 


.795 

.755 


.755 

.699 

»Zxi = 

.699 

.626 

.626 

.532 


.532 

0 


.418 

0 


0 

0 


0 

0 


0 


1.089 I .418 

7.584 


= .199 


4. hj=(R—\)i (ajh^bx-bi-i H-1 a„bibx-b„-\) 

5. ^ _ a\ ± ajPx ± —h (aj-\b\b2-&j-2) - 1 

” J ajb\b2-bi-\ H-h 

Exercise Set 11.20 (page 767) 


1 _2 4 

-I- 4 cos t V cos 2t + ^ cos 3t 
3 y 


T 2 I F 2 / 2x, I 1 4ff, . 1 6k f . 1 rrt 8 tt, 

I — | coS“^"i I ~ cos i I ~ cos _ t | ~ cos i 

3 2 7 4 


3 * 2 


T 2 - 


■-^(sin^-f 4 | sin^ 4 ^ sin^ 4- ^ sin ^-t 


4 4 4- sin£ — tt - cos — —i - cos 4£ 

^2 3tt 15?r 


— [77 — cos t ~ cos 2t ~ T^T cos 3 ^ — ‘ 
* \ 2 1-3 3-5 5-7 


1 


( 2 »- 1 )( 2 »+ 1 ) 


COS fit 


T 8Y ( 1 2frt . 1 6 tt^ . 1 lO^T . . 1 2 n~t 

— —- 1 COS —— I —r - COS —— I — COS- I — I -— COS——— 


4 tP \2 2 


6 " 


10" 


( 2 »)' 


Exercise Set 11.21 (page 775) 





















1. 


(a) YeS;v = |vi+fv 2 + |y 3 

(b) No; v=| v H fv 2 -Iv 3 

(c) Yes; v = f V1 f f V2 + ° V3 


(d) 


Yes; y — J-v, 4 —v? 4- — 

15 1 15 2 15 3 


m - number of trainees = 7, n = number of vertex points = 1, k = number of boundary vertex points = 5; Equation (7) is 
7 = 2(7) -2-5- 


vr— Mv + b — M{c\\\ + C2V2 + C3V3) + (^1+^2 + C3)b 
= c\{Mx\ + b) 4 - C2(M v 2 + b) + C3(Jlfv3 4 - b) 

= C 1W) -1- C /Vf'/ + C 3 W 3 





















7, 


Two of the coefficients are zero 


(a) 


At least one of the coefficients is zero 

(b) 


None of the coefficients are zero 
(c) 


8 . 


(a) 3 


vi 


4- j\'2 + jv 3 


(b) 


8/3 

2 
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